Using YQL Execute to power the Pipes Webservice Module -updated

YQL has become a great way to extend Pipes. You can harvest the power of YQL by using YQL statements in the YQL module. Another way of using even more of YQL’s power is to use the Pipes Webservice module. This enables you to use YQL’s Execute functionality to create javascript functions that will work on your data in any Pipe.

In this blog post we’ll go through the steps on how to use YQL Execute to power the Pipes Webservice Module – for the impatient, here’s the example Pipe that we’ll be creating.

In order to use YQL Execute, you’ll need a website to host your XML file. The XML file is where YQL Execute allows you to manipulate data by using server side javascript. We can then post data to YQL via the Pipes Webservice module to allow further processing of our Pipes Data.

Some users on the Pipes message boards asked if we could add a “yindex” element that numbers each item in the output. This becomes trivial when using YQL Execute!

The first thing we will create is our XML file for YQL Execute. You can copy and paste the structure of the XML below, editing only whats inside the execute node. yqlexecuteupdated
The execute content is wrapped within a CDATA statement. Since we’re dealing with json, we’ll copy Crockfords json parser and stringifier to our local server and import it using the y.include() method. First, we parse the json data using JSON.parse(), then loop through the json structure and add new elements by using pdata.items[i].new_element_here within the loop. “yindex” is the new item we want added to our Pipes data structure. We then assign a number to yindex within the loop. Finally, we assign our “pdata” variable as our response.object. Then we want to host this XML file on a publicly accessible web host or host it on YQL using YQL storage (http://developer.yahoo.com/yql/guide/yql-cloud-chapter.html).

Now we’ll set up our Pipe. We need to use the URL builder to setup the URL that the Webservice module will access.

In the URL Builder module, we’ll use

  • http://query.yahooapis.com as our base
  • v1/public/yql as our Path element
  • These name/values as our query params
    • q: use “http://paul.donnelly.org/yql/yindex.xml” as yindex; select * from yindex where data = @data
    • format: json
    • diagnostics: false

And simply wire the URL builder module into the url terminal in the Web Service module. Be sure to use query.results.result.items as the Path to item list.

Here is the example Pipe that takes the Pipes blog feed, uses YQL execute as a Webservice module to create an additional element in the Pipes data output: “yindex” which adds a number to each item in the Pipes results. Another example YQL Execute statement is: use “http://paul.donnelly.org/yql/random.xml” as ran; select * from ran where data = @data that randomizes a given feed.

This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>