Pipes in Text

A few days ago, Stefan joined a rant of Peter Williams on Yahoo! Pipes lack of a text representation:

While for many models (and programs, and anything in between) having a visual representation is nice when you want to read (or view) it, visual authoring sucks in the vast majority of cases. Sadly, being able to efficiently edit something in a text editor, with versioning and diff support and so on is in general not what impresses those who make purchasing decisions.

This is so true.

One more day, and he links to a great presentation on Pipes and the Y! Query Language. Says Abel Avram in the post that directs to the presentation by Jonathan Trevor , captured at QCon SF 2008 (53 mins):

Yahoo Pipes uses a visual tool to specify a series of pipes through which input data flows and is filtered and processed in various ways. There are many data sources like personal data, CSV, feeds, web pages, services like Flick, Yahoo Search, and others. Processing is done through operators like: Filter, Loop, Regex, Sort, Union, and others. The result is shown in a web page, a feed, or given to an application. The service runs in Yahoo’s cloud, is free and does not offer security protection for sensitive data since anyone can copy and run a pipe made by someone else.

YQL is similar to Pipes but uses a textual textual language and can process both Yahoo web services data and any structured data with an URL. It has an SQL like syntax with three statements: SELECT, SHOW, DESC. This approach is more powerful and the applications created are protected.

Thanks, this is what a lot of people are looking for.