My 2 cents on Foreign Keys and Data Warehousing
As a data warehouse architect/designer, we always have to deal with the question: “should I use foreign keys ? Should I enforce them ?” Well, yes! for the first question, probably not for the second. Here is why.
Creating an execution dashboard for Talend jobs
Talend Open Studio is a great open source solution. Its “Cost vs. functionalities” is quite good. But because this is the free version, as opposed to the enterprise edition, it lacks a few core functionalities such as a proper workflow manager or a simple execution dashboard. But as it turns out, creating an execution dashboard is actually quite easy.
Hierarchical and recursive queries in (T-)SQL
“Hierarchical and recursive queries in SQL”… this is quite a hot topic, so hot that it has its own wikipedia entry. The typical “hello world” is employees and managers.
Incremental data loading with Talend
One of the inevitable challenge in any data integration architecture is choosing the right loading technique. Push change data capture (CDC) is more than often not available. Full (re-)load, batch delta/daily load and incremental load (CDC pull) are potential candidates. In this post, I will describe a possible implementation of an incremental load using Talend. Only the relevant part for CDC will be described (no processing of the data).