Lambda Architecture, Master Dataset and Data Vault
The Lambda Architecture got known after Nathan Marz' and James Warren's book about Big Data. The authors describe a data processing architecture for batch and real-time data flows at the same time. Fault-tolerance and the balance of latency vs throughput are main...
SQL – Only the Strong Survive
There are tons of tech predictions at the end of every year. Eric Knorr lists "The incredible SQL comeback" in his 9 enterprise tech trends for 2017 and beyond. An old, mature programming language as the prevailing trend for 2017 and later?After years of NoSQL...
GOODS – How to post-hoc organize the Data Lake
Google describes in a recent paper „Goods: Organizing Google’s Datasets“ their approach for post-hoc metadata management. GOODS (GOOgle Dataset Search) is a system that crawls internal storage systems (e.g. GoogleFS, Bigtable, DBMSes) in order to collect, aggregate,...
Is my Oracle database healthy?
It's a common question if a database is in a healthy state. DBA_REGISTRY shows the status of components loaded into the database. Mike Dietrich mentions in his blog two additional checks that should be performed regularly during off-peak hours for Oracle...
Data Vault 2.0, Hashing and DB2 LUW
Hashing is part of Data Vault 2.0. Hash Keys are used to provide a surrogate for business keys detect changes in fields in Satellite tables. Oracle, SQL Server, Informatica, etc have built-in functions to compute a hash key. Example with Oracle: select...
Amazon Redshift (hosted DWH) – SQL for simplicity
NoSQL databases like Amazon Dynamo got quite popular in the OLTP market. Their marketing credo is supposed to be simplicity compared to RDBMS. The absence of a declarative programming language like SQL means that a lot of work has to be done in the application though....
DataBeat week 06/2015
Reference to blogs, tweets, discussions, etc that caught my attention during the last week. Data Modeling "Data Modeler Of 2020" YouTube video by Steve Hoberman addresses topics like modeling for NoSQL or BigData, Fact Based Modeling, becoming more agile, etc. "Basic...
DataBeat week 05/2015
Reference to blogs, tweets, discussions, etc that caught my attention during the last week. Data Architecture "Immutability changes everything" by Pat Helland deals with immutability as a key architectural concept. Append-only computing instead of changing data is...
DataBeat week 04/2015
Reference to blogs, tweets, discussions, etc that caught my attention during the last week. Data Storage "Introducing Espresso - LinkedIn's hot new distributed document store" by Aditya Auradkar contains a detailled introduction (data model, architecture, key...
DataBeat week 03/2015
Reference to blogs, tweets, discussions, etc that caught my attention during the last week.Data Architecture"Hyper-threading – how does it double CPU throughput?" by Aurimas Mikalauskas shows some interesting tests with Hyper-threading turned on and provoked some...
DataBeat week 02/2015
Reference to blogs, tweets, discussions, etc that caught my attention during the last week.Data ModelingSteve Hoberman asked the question "Create a logical data model?" on his blog and supplied his and responder's opinions. There is an overall agreement that a logical...
Data Vault resources
An overview of Data Vault (data modeling / architecture / methodology) resources - no claim to be complete - order of entries below are arbitrary Books The Official Data Vault Standards Document (Version 1.0) (Data Warehouse Architecture), Dan Linstedt, 2012...