Re: BIG data architecture

Posted by Roshan on
URL: http://erman-arslan-s-oracle-forum.114.s1.nabble.com/BIG-data-architecture-tp8559p8565.html

Hi Erman,

After GoldenGate extracts data from source and replicates to our BI environment, we use SQL/PLSQL scripts which are scheduled on crontab to process data using the tables replicated and load into a temporary table. The issue is as data grows(tables), the processing time wil increase and more resource will be needed.

I would like to connect the hadoop with streamset to our BI platform to prevent us using SQL/PLSQL scripts.

BI servers (on Solaris and RAC) --- streamset --- HADOOP.
https://streamsets.com/

Second, you mentioned "PLSQL transformation is cheap but it should be managed well. It should be under control of some sort of application.. Else, you will do lots of maintanence."

Please advise which sort of application should we use?

Thanks,

Roshan