String aggregation with databases and ETL tools
String aggregation is the process of concatenating strings (usualy as a list with a separator) based on a common key. Databases implement their own function to do so such as wm_concat or LISTAGG in Oracle, group_concat in MySQL, array_agg or string_agg with PostGres. Some databases doesn’t support it at all, such as MS SQL Server (before the 2017 version) and require workaround. The implementation on the ETL tools side also depends on the vendor.
Variables declaration ordering error in Informatica and Talend
Understanding how in-flow variables are declared by your ETL tool is important. Probably all of them requires the user to declare the variables in the right orders. But what happens when there is an ordering error in a component of your flow.
My 2 cents on Foreign Keys and Data Warehousing
As a data warehouse architect/designer, we always have to deal with the question: “should I use foreign keys ? Should I enforce them ?” Well, yes! for the first question, probably not for the second. Here is why.
Creating an execution dashboard for Talend jobs
Talend Open Studio is a great open source solution. Its “Cost vs. functionalities” is quite good. But because this is the free version, as opposed to the enterprise edition, it lacks a few core functionalities such as a proper workflow manager or a simple execution dashboard. But as it turns out, creating an execution dashboard is actually quite easy.