Long long time ago Data Engineering was considered a boring work by software engineers. SQL ruled everywhere and Oracle and Microsoft were leaders with their products and PF Codd was worshipped like a God. It was a peaceful time but with data-constraints and principles based relational theory. But the greed of speed for business was unfathomable.They wanted more data , more velocity ,deeper insights and most importantly the desire to see the future which was very unpredictable and called as "Predictive Analytics". Then came the open source vikings. They denounced the SQL as unholy worship and NoSQL became their Chant.They were experienced warriors in OOPS primary with Java as their weapon of choice. Their first attempt of taking the seige of the wonderful Data world had failed with Object Oriented Databases so now they came with NoSQL to disrupt the peaceful life that Data engineers had build with massively successfully OLTP ,OLAP and BI platforms using time tested principles and techniques.They had the beautiful Cassandra and the cross breed child Mongo. They were joined by poor peasants with no programming experience to get a livelihood and get a better life. But the worst who joined them were the war mongoring MBAs who had no SKIN IN THE GAME. The MBA's were worst of all. They only had opinions as weapons from reading blogs and articles but they made a lot of noise using blogs and articles quoting HBR , New Yorker Economist and other.Data Engineers folks decide to defend their territory they went to SPARK learned a new weapon called SCALA that could stop the tide of Java army of warriors. Now the Data World is in conflict with expensive delays with no principles.Business is questioning the ROI but they still falsely believe Data Hoarding is Big Data. The only is hope is SPARK can bring peace to the land so Software Engineers and Data Engineers can live together in harmony. Only time will tell what will happen......
No comments:
Post a Comment