DigData_logo

The changing world

Traditional ETL-based data integration has been recently challenged by the rapid growth of data volumes. The immerse of the new Hadoop-based data storage and processing techniques has addressed that need, but at a cost of taking us back to the time of manual hand-(and hard-) coding techniques. In the world where change is the only constant our ability to efficiently maintain and modify the data related processes is a key requirement, which we cannot forget about. How should we then marry the best of the two worlds – the ease of development and maintenance of ETL with the capabilities of modern Hadoop and appliance environments, or simply the underutilized capabilities of our RDBMS systems?

What is DigData

DigData is a modern data integration solution, combining the best of various approaches to the data processing and integration. It allows you to graphically develop your processes, and execute them on the Hadoop, appliance or RDBMS of your choice.

DigData1

Key features:

  • Graphical development environment
  • Majority of the cost related to your data integration is related to the time it takes to build and maintain the particular processes. Reduce it by using an intuitive, easy to use GDE, without scarifying the performance or flexibility of the final solution.

  • Execution on a variety of platforms
  • Once you have build your processes you can run them on multiple platforms – Apache Hadoop, IBM BigInsights, Cloudera, Hortonworks, Teradata, DB2 or simply an Oracle database (more platforms to come). This also means that you can migrate from one to another without the rewriting efforts usually related to platform changes.

  • Optimal execution
  • DigData allows you to execute various phases of a single process on different environment. You can filter and join your data at the source pushing this phase to the source database or Hadoop instance, perform complex transformations in a highly parallel Hadoop cluster, and do final data formatting in the target of your choice.

  • Built-in data lineage
  • Data integration in DigData comes with field level data lineage fully integrated with all major metadata repositories (IBM Information Governance Catalog, Informatica Metadata Manager, Ab Initio Metadata Hub). Your end to end data lineage will simply extend to the DigData processes as the next step of the processing pipeline.

  • Automatic import of existing processes
  • With DigData your team of ETL developers can keep doing what they are best at – developing your data integration processes, utilizing their knowledge of the data and the environment. The implementation details of the underlying Hadoop environments is hidden under the cover, allowing the developers to focus on the data and the processing logic rather than on the specifics of the underlying data platform.

  • Simple infrastructure
  • To use DigData you do not need tons of new hardware. The DigData Server can run on a tiny physical or virtual server as it does not perform any heavy data transformations. The actual execution is pushed to your database or Hadoop systems, giving you full control on the load you generate on every of the engaged environments.

Develop in ETL & run on Hadoop

With DigData your team of ETL developers can keep doing what they are best at – developing your data integration processes, utilizing their knowledge of the data and the environment. The implementation details of the underlying Hadoop environments is hidden under the cover, allowing the developers to focus on the data and the processing logic rather than on the specifics of the underlying data platform.

DigData2

Learn more

DigData Product Collateral

To find out more about how we can help you please contact us