Building an ETL Pipeline with Open Source Tools

What Is ETL

ETL stands for Extract, Transform, Load. Extraction is the process by which data from many sources and formats is collected. The data is then processed to allow for ease of storing and future processing. This can include data cleaning, or format normalization into file structures such as JSON. From here the data can then be persisted for storage and access by interested stakeholders.

Continue reading

Yes, Rust has Garbage Collection, and a Fast One

Rust is getting more and more popular. Thus, a group of colleagues, including myself, evaluated it for half a day to build up our own opinion, if Rust is of strategic interest for us or not. We did some coding following the standard introduction book, looked at some frameworks, and watched the presentation “Considering Rust”. The general conclusion was more or less in the line of: yeah, a nice new programming language, but without a full-grown ecosystem and without any garbage collection, it will be too cumbersome and unproductive for us in our projects. My gut feeling did not agree with the assessment regarding garbage collection. Thus, I did some more digging and testing and came up with my current conclusion: Rust does indeed garbage collection, but in a very clever way.

Continue reading