3 Key Steps to Break Free from Legacy ETL

Video Transcription:

If you got Databricks, at least you have broken your silos of your DB2s and Teradata and Hadoops and the Cloud Data Warehouse into one platform. But what about all these lines? What about all this ETL? Have you modernized that?

That is the real question. Just getting to Databricks wouldn't solve your problem, so this is where our partner Prophecy comes into picture.

What they help us do is rationalize and modernize all the siloed ETL into one layer.

Most companies, if they are used to GUI based ETL tool, right, because it's so easy to develop, then Prophecy is a natural fit.

It provides you a nice GUI interface. But with just one click of a button, you can even see all the PySpark code and you can even change the code and it gets reflected back into the GUI.

So it's really a two way product where you can do GUI based development, which is great for your citizen data analyst, but your core data engineering team is also happy because they can do genuine performance tuning by providing hints into Spark, etc.

You don't want even your line of business teams, right, all your finance marketing teams to be siloed as well.

So you want all your data analysts and citizen analytical engineers to also use the lakehouse platform, get access to all the data.

So for that, it's very important that you create agile teams, give all your teams access to all the data in one lake house model, and you enable them with easy tools like Prophecy.

So it even helps you get more users to use all your data using this tool, right? So essentially, you are now getting more users to do ETL and analytics, right? The ELT model, where central IT teams are doing the ingestion enterprise cleansing. And then instead of five people or ten people who are core data engineers, you are using 50 people in your enterprise.

So expose more analytical engineers to this architecture, and this, of course, unlocks your AI initiatives.

So we do have a joint program with Prophecy. I'm, by the way, from the migrations team in Databricks. So we have a joint program with Prophecy, where we can come we can analyze all your xml's of your legacy ETL, very quickly figure out the work, and then create a modernization plan for you.

‍