Workflow Engines and Building a Domain Specific Language for Data Quality with Tom Baeyens

Open||Source||Data

This episode features an interview with Tom Baeyens, Co-founder and CTO of Soda, where he oversees the company's product development, software architecture, and technology strategy. He is passionate about open source and committed to building a community where data engineers can succeed using the Soda Data Monitoring Platform. Tom is the inventor of the widely-used open source project jBPM and Activiti. He also co-founded Effektif, a cloud process automation company.

In this episode, Sam and Tom discuss the evolution of open source workflow engines, data contracts, and why data quality needs a language approach.

-------------------

“Where we're heading is what I think is exactly the same as with software engineering in the testing. Test-driven development was a radical new thing back then. But then it turns out, you can much more reliably release software. And this is exactly the same here. If you don't inject data testing, data observability throughout your data stack, then how are you going to trust the data that you put into your machine learning model? This is something that people are realizing, but we're still figuring out the best practices, the dos, the don'ts. We've come a long way, but there's still a way to go before this is as common and as normal as in the test-driven development software engineering space.” - Tom Baeyens

-------------------

Episode Timestamps:

(01:23): What open source data means to Tom

(04:34): Tom’s motivations for creating jBPM

(09:39): What led Tom to building Soda

(13:57): Why data quality needs a language approach

(19:24): The community of Soda

(22:47): The future of Soda as a technology

(24:59): A question Tom wishes to be asked

(30:24): Tom’s advice for engineers who want to leverage data observability tools

-------------------

Links:

LinkedIn - Connect with Tom

Twitter - Follow Tom

Visit SodaCL