May 21, 2014
Dan Kusnetzky

http://kusnetzky.net/virtual-worlds/concurrent-releases-cascadi.html
http://www.zdnet.com/concurrent-launches-cascading-3-0-7000029718/

From time to time the folks at Concurrent, Inc. let me know about improvements to its technology or about new products. This time, Cascading has been enhanced to support Mapreduce and Tez.

What Concurrent has to say about Concurrent 3.0
Cascading 3.0 Sets the Standard for Enterprise Application Development

With more than 150,000 user downloads a month, Cascading is the de facto standard in open source application infrastructure technology. Supported by key strategic partnerships with Hortonworks and Databricks, and broad support with all major Hadoop distributions, Cascading is the enterprise development framework of choice for data-centric applications. Cascading accelerates and simplifies enterprise application development, and meets a variety of enterprise use cases from simple to complex.

Cascading 3.0 is a major leap forward in enterprise data-centric application development. Features and benefits include:

• Cascading 3.0 provides the most comprehensive data application framework to meet business challenges and solve a variety of business problems ranging from simple to complex, regardless of latency or scale.
• Cascading 3.0 allows enterprises to build their data applications once, while providing the flexibility to run applications on the fabric that best meets their business needs.
• Cascading 3.0 will ship with support for: local in-memory, Apache MapReduce, and Apache Tez.
• Soon thereafter, with community support, Apache Spark™, Apache Storm and others will be supported through its new pluggable and customizable query planner.
• Third party products, data applications, frameworks and dynamic programming languages built on Cascading will immediately benefit from this portability.
• Cascading offers compatibility with all major Hadoop vendors and service providers: Altiscale, Amazon EMR, Cloudera, Hortonworks, Intel, MapR and Qubole, among others.

Snapshot analysis
I’ve written about Concurrent before (see Concurrent Driven: Big data application performance management for more information). This announcement focuses on allowing Cascading users to develop data-focused applications for both the Apache Mapreduce and Apache Tez environments as well as other Big Data platforms the company currently supports (all major Hadoop vendors and service providers: Altiscale, Amazon EMR, Cloudera, Hortonworks, Intel, MapR and Qubole, among others).

The company’s goal clearly is to make its application framework a necessary part of Big Data application development. Concurrent has promised to integrate its application framework with Apache Spark and Storm in the near future.

If your company is developing Big Data applications, Concurrent should be on your watch list.

May 14, 2014
Angela Guess

http://www.dataversity.net/concurrent-inc-leads-market-data-driven-enterprise-application-development/

According to a recent article out of the company, “Concurrent, Inc., the enterprise data application platform company, today announced product and corporate momentum securing the company’s leadership in enterprise application development. The company recently announced strategic industry partnerships with Hortonworks and Databricks, as well as new product innovation with the introduction of Driven, the industry’s first application performance management product for data-centric applications. Today Concurrent also introduced the next version of Cascading, the most widely used application development framework for building data applications on technologies like Apache Hadoop.”

The article continues, “Enterprises have always been operationalizing their data. But as business needs continue to change and new technologies – such as Apache Hadoop and now Apache Tez – emerge, organizations need a reliable way to quickly build and consistently deliver these data products. This requires leveraging existing skill sets, while meeting new requirements (i.e. latency, scale, service level agreements) supported by these emerging technologies. With more than 150,000 user downloads a month, Cascading is the de facto standard in open source application infrastructure technology. Supported by key strategic partnerships with Hortonworks and Databricks, and broad support with all major Hadoop distributions, Cascading is the enterprise development framework of choice for data-centric applications. Cascading accelerates and simplifies enterprise application development, and meets a variety of enterprise use cases, from simple to complex.”

Apr 27, 2014
Maria Deutscher

http://siliconangle.com/blog/2014/04/27/what-you-missed-in-big-data-hadoop-applications-watson-at-the-forefront/?angle=silicon

Data-driven applications returned to the headlines this week after Hortonworks announced that it will bundle the open source Cascading development framework into its flagship Hadoop distribution. Created and maintained by a company called Concurrent, Cascading is a Java-based abstraction layer that allows users to take advantage of the batch processing platform without mastering MapReduce or even changing the way they work.

Cascading supports a broad range of enterprise technologies, including Java, SQL and a number of popular data science tools. Future releases of the version, which the Yahoo! spin off has committed to certifying as part of its partnership with Concurrent, will also feature integration with the emerging Apache Tez Hadoop query framework.

Apr 25, 2014
Jeffrey Kelly

http://siliconangle.com/blog/2014/04/25/the-data-economy-meet-the-hybrid-data-scientist-application-developer/

Hadoop meets application development tooling

On the tools set side of the equation, Hortonworks recently expanded its partnership with Concurrent, which sells support services for the open source Cascading application development framework. When I spoke with the company last fall, Concurrent Founder and CTO Chris Wensel described Cascading as a Java library used by application developers to quickly create complex, data oriented applications. Concurrent’s Cascading SDK abstract’s away the complexity of dealing with things like MapReduce and Pig, allowing developers to integrate data sources via APIs and easily migrate predictive models into Hadoop. (You can explore sample Cascading-based apps on GitHub here.)

HDP with Cascading (Source: Hortonworks)

As part of the expanded partnership, Hortonworks said it will ensure ongoing compatibility of Cascading-based apps with the Hortonworks Data Platform and will provide level 1 and level 2 Cascading support for customers (Concurrent will still handle level 3 support.) This compatibility includes the ability to execute Cascading-based apps on Apache Tez, a recently developed Hadoop-based execution engine for real-time Big Data workloads. While Concurrent itself is still in its early days, open source Cascading is quite popular with application developers, garnering over 90,000 downloads per month.

Apr 24, 2014
Richard Harris

http://appdevelopermagazine.com/1354/2014/4/14/Concurrent-Announces-New-Capabilities-for-its-Application-Development-Framework-for-Data-Applications-on-Hadoop/

Concurrent, an enterprise data application platform company, and Hortonworks, a provider of enterprise Apache Hadoop, have announced the Concurrent Cascading SDK will now be integrated and delivered with the Hortonworks Data Platform (HDP). In addition, Hortonworks will certify, support and deliver the Cascading application development framework for data applications on Hadoop.

Hortonworks will certify, support and deliver the Cascading SDK with HDP guaranteeing the ongoing compatibility of Cascading-based applications across future releases of HDP with continuous compatibility testing and direct HDP customer support for Cascading.

Cascading will now also support Apache Tez which enables Hadoop projects to meet demands for faster response times and delivering near real-time big data processing. Tez is a general data-processing fabric and MapReduce replacement that provides a framework for executing a complex topology of tasks. In addition, Tez executes on top of Apache Hadoop YARN, a sub-project of Hadoop which separates resource management and processing components. YARN fundamentally enables a broader array of interaction patterns for data stored in HDFS beyond MapReduce and makes Hadoop 2.0 a more general data processing platform.

Companies that already use Cascading, Lingual, Scalding or Cascalog, or any other dynamic programming language APIs and frameworks built on top of Cascading, can now migrate to newer versions of HDP that support Apache Tez, with zero investment required to take advantage of this improved processing environment.

Apr 23, 2014
Arnal Dayaratna, Ph.D

http://cloud-computing-today.com/2014/04/23/concurrent-and-hortonworks-partner-to-integrate-cascading-with-the-hortonworks-data-platform/

Concurrent and Hortonworks recently revealed a deepening of their strategic relationship whereby Cascading SDK will now be integrated into the Hortonworks Data Platform. Moreover, Hortonworks will certify, deliver and support Cascading, the application framework for developing Hadoop-based applications. A Java-based, open source alternative to MapReduce, Cascading provides developers with a framework for constructing complex, repeatable data processing tasks within a Hadoop cluster. Cascading features an abstraction platform which uses plumbing metaphors such as taps, pipes, data flows, cascades and sinks to allow developers to design, visualize and execute jobs and processes on Hadoop-based data without having to master the intricacies of MapReduce. Forthcoming releases of Cascading will support Apache Tez, an initiative that represents the next step after the addition of YARN to Hadoop that allows for Hadoop-based data to “meet demands for fast response times and extreme throughput at petabyte scale.” The partnership between Concurrent, the developer of Cascading, and Hortonworks, represents a huge coup for Concurrent given that the collaboration stands to rapidly accelerate Cascading’s adoption in enterprise environments. Hortonworks, meanwhile, benefits from packaging its Hadoop distribution with Cascading, one of the industry’s most well respected frameworks for Big data management and application development that boasts enterprise users such as Twitter, LinkedIn, eBay and Nokia. The obvious question now is whether Concurrent will finalize similar partnerships with other Hadoop vendors such as Cloudera and MapR or whether Concurrent’s partnership with Hortonworks enables the latter to improve its positioning in the battle for Hadoop market share, particularly in light of Cloudera’s remarkable $900 capital raise and partnership with Intel.

Apr 23, 2014
David Ramel

http://adtmag.com/Articles/2014/04/23/hortonworks-concurrent-partnership.aspx?Page=1

Hortonworks Inc. and Concurrent Inc. announced this week they are partnering to make Hadoop development easier and quicker by combining the former’s data platform with the latter’s Cascading application development framework.

As part of the expanded partnership, Hortonworks, a leading force in Hadoop development, will certify, support and include the open source Cascading SDK with the Hortonworks Data Platform (HDP). Also, future Cascading releases will support Apache Tez, an alternative to the original MapReduce programming model used with Hadoop that has often been criticized for a slow, non-interactive, batch-processing model. Tez supports more interactive queries, faster response times and extreme throughput at huge scales.

“Users of Cascading will now be able to rapidly build data-centric applications that take advantage of the Tez providing users with highly interactive operational applications on top of Hadoop,” said John Kreisa in a Hortonworks blog post announcing the partnership.

Kreisa claimed that Cascading is “the most widely used application development framework for data applications on Hadoop.” He said Hortonworks will guarantee “the ongoing compatibility of Cascading-based applications across future releases of HDP with continuous compatibility testing and direct HDP customer support for Cascading.” The companies will team up to provide different levels of support to Cascading users.

Hortonworks and Concurrent yesterday presented a webinar in which they described how to accelerate Big Data development with Cascading and HDP.

In the webinar, Concurrent executive Supreet Oberoi explained the genesis of the Cascading framework produced by his company, which was founded in 2008 by Chris Wensel, a pioneer in the Hadoop phenomenon who started the first Silicon Valley meetup for the budding technology.

Wensel thought Hadoop was powerful, Oberoi said, but he saw some challenges with the technology.

“The first one was that people who are used to developing data applications, they think in terms of business objects, and making them think in terms of maps and reduce is unintuitive,” Oberoi said. “The second point is that even though these APIs were written in Java, the level of complexity will preclude many of the existing Java community developers from using those APIs.”

The third, Oberoi continued, was the realization that different use cases require different execution fabrics, and being coupled to one prevented developers from using better or more appropriate technologies that might come along in the future. With those reasonings, Wensel developed the Cascading API.

“As a Java-based framework, Cascading fits naturally into JVM-based languages, including Scala, Clojure, JRuby, Jython and Groovy,” noted this site’s editor at large John K. Waters in a blog post last winter. “And the Cascading community has created scripting and query languages for many of these languages.”

Jules S. Damji explained in a Hortonworks blog post Monday that Wensel developed the API “with the sole purpose of enabling developers to write enterprise big data applications without the know-how of the underlying Hadoop complexity and without coding directly to the MapReduce API. Instead, he implemented high-level logical constructs, such as Taps, Pipes, Sources, Sinks, and Flows, as Java classes to design, develop, and deploy large-scale big data-driven pipelines.”

Damji further explained how to get started with Hadoop and Cascading with some simple examples and pointed developers to this tutorial for more detailed information.

Apr 23, 2014
Pam Baker

http://www.fiercebigdata.com/story/big-data-app-development-brought-you-hortonworks-cascading/2014-04-23?utm_source=rss&utm_medium=rss

Like in other technologies, there’s a huge need for applications and big data is no exception. Subsequently there is a need to assist developers in getting such to market and in play quickly. To that end, Hortonworks has added Concurrent’s Cascading SDK to its Hadoop distribution. Such helps developers operationalize their data. In addition, Hortonworks will certify, support and deliver Cascading–the most widely used App development framework for data applications on Hadoop.

“As more enterprises realize they are in the business of data, the need for simple, powerful tools for big data application development is a must-have to survive in today’s competitive climate,” said Gary Nakamura, CEO, Concurrent, in a statement to the press. “Our deepened relationship with Hortonworks furthers our commitment to Hadoop and drives new innovation around the development of enterprise data applications.”

Upcoming releases of Cascading will also support Apache Tez, a general data-processing fabric and MapReduce replacement that provides a powerful framework for executing a complex topology of tasks. According to the press release:

“Tez executes on top of Apache Hadoop YARN, a sub-project of Hadoop which separates resource management and processing components. YARN fundamentally enables a broader array of interaction patterns for data stored in HDFS beyond MapReduce and makes Hadoop 2.0 a more general data processing platform.

In addition, thousands of companies that already use Cascading, Lingual, Scalding or Cascalog or any other dynamic programming language APIs and frameworks built on top of Cascading, have the flexibility to seamlessly migrate to newer versions of HDP that support Apache Tez, with zero investment required to take advantage of this improved processing environment.”

It will be interesting to watch how quickly developers take to this and how many new apps show up as a result. My bet is it will be plenty.

Apr 22, 2014
John Rath

http://www.datacenterknowledge.com/archives/2014/04/22/hortonworks-and-concurrent-partner-to-speed-data-centric-application-development-on-hadoop/

Hortonworks and Concurrent announced an expansion of a strategic partnership to simplify enterprise application development for data-centric applications. Hortonworks will certify, support and deliver Concurrent’s Cascading development framework for data applications on Hadoop, and the Cascading SDK will be integrated and delivered with the Hortonworks Data Platform (HDP).

The partnership underscores the timely importance of simplifying enterprise application development for these new data-centric applications. It benefits users by combining the robustness and simplicity of Cascading with the reliability and stability of Hortonworks Data Platform.

Upcoming releases of Cascading will also support Apache Tez. Tez is a significant development in the Hadoop ecosystem, enabling projects to meet demands for faster response times and delivering near real-time big data processing. Tez is a general data-processing fabric and MapReduce replacement that provides a powerful framework for executing a complex topology of tasks. In addition, thousands of companies that already use Cascading, Lingual, Scalding or Cascalog, or any other dynamic programming language APIs and frameworks built on top of Cascading, have the flexibility to seamlessly migrate to newer versions of HDP that support Apache Tez, with zero investment required to take advantage of this improved processing environment.

“Hadoop unleashes insight and value from enterprise data as a core component of the modern data architecture, integrating with and complementing existing systems,” said John Kreisa, vice president of strategic marketing at Hortonworks. “By expanding our alliance with Concurrent and integrating with the Cascading application platform, Hortonworks’ customers can now drive even more value from their enterprise data by enabling the rapid development of data-driven applications.”

“As more enterprises realize they are in the business of data, the need for simple, powerful tools for big data application development is a must-have to survive in today’s competitive climate,” said Gary Nakamura, CEO at Concurrent. “Our deepened relationship with Hortonworks furthers our commitment to Hadoop and drives new innovation around the development of enterprise data applications.”

Apr 22, 2014
Maria Deutscher

http://siliconangle.com/blog/2014/04/22/hortonworks-boosts-concurrent-team-up-for-big-data-applications/

The elusive promise of the Big Data app economy has inched a little closer to reality on Monday after Hortonworks expanded its partnership with Concurrent to package the startup’s Cascading development framework into its flagship Hadoop distribution.

Available for free under an Apache license, Cascading serves as an abstraction layer between the batch processing platform and the applications that use it, allowing enterprise developers to tap into their organizations’ vast troves of unstructured information without getting bogged down by the inherent complexity of MapReduce.

“Building applications on top of Hadoop was very difficult. That’s why our founder Chris Wensel created a framework so you could have a separate business logic layer from the data layer, and it’s written in Java so any Java programmer can pick it up,” Guy Nakamura, the CEO of Concurrent, told SiliconANGLE in an exclusive interview on theCUBE at O’Reilly Fluent Conference 2013.

Cascading goes above and beyond just making it easier to create data-driven applications, completely eliminating the need for users to change the way they work through support for broad range of enterprise technologies, including SQL and a number of popular data science tools. “The requirement for the enterprise is not to learn new skills for Hadoop but to leverage existing skills, existing systems and existing investments they already made in their infrastructure,” Nakamura explained. Upcoming versions of the framework will also include integration with Apache Tez, an emerging alternative to MapReduce that aims to deliver better performance and lower latency at large scale.

Tez runs on top of the YARN resource management and scheduling technology included in Apache Hadoop 2.0, which constitutes the core of the latest Hortonworks Data Platform (HDP) 2.0. Under the expanded partnership, the distributor is “guaranteeing the ongoing compatibility of Cascading-based applications across future releases” and offering customers dedicated support for the framework.

Boosting business

The partnership makes sense for both companies. Hortonworks is coming under increased pressure to deliver value higher up the stack and enabling applications on top of Hadoop is one of the best possible ways of accomplishing that. Plus, the integration allows it to catch up with rivals Cloudera and MapR, which have long provided support for Cascading in their respective distributions.

The announcement is also good news for Concurrent. The company’s flagship framework is now compatible with all three major Hadoop distributions, making it easily accessible to the overwhelming majority of users. The partnership with Hortonworks is especially significant because the two firms have very similar business models: they both make their their flagship products available at no charge and and monetize their user bases through value-added solutions. But whereas the Yahoo! spin-off focuses exclusively on professional services, Cascading sells complementary software such as its recently released Driven application performance management tool. Free while in beta, the cloud-based service provides visibility into data flows and program logic at runtime to enable test-driven development while allowing practitioners to keep tabs on information quality, according to the firm.