Showing posts with label google cloude. Show all posts
Showing posts with label google cloude. Show all posts

Friday, January 3, 2020

Cloudera cluster creation on google compute instance

Wanted to quickly launch my cloudera cluster in google cloud(since i had some free credit wanted to try effectively)  similar to on prem cluster like single sign  on  all the linux nodes.

initially i though of integrate all linux servers with Active directory + SSSD client but later i moved to MIT kerberos + Open LDAP client + SASL passthrough.

This script has 2 part

PART 1: will create No. of gcp instances, create hadoop users/groups, install SASL/openldap/MIT kerberos/Cloudera agent and Manager

PART 2: Will add the hosts into cloudera manager,create cluster/add hdfs and zookeeper services.(still working on adding more services)

Part 2 of this script can be easily  scale up with any cloud providers(AWS,AZURE) as long as cloudera manager url  is exposed to internet.