Although cloudera offer cluster utilization report which will provide detailed report about Yarn utilization, there are some limitation. so i used my scripting knowledge to implement on my own.
please find below python and script and the email output.
From:
ClouderaAdministrator-
Sent: Thursday, April 2, 2020 8:00:01 AM (UTC+00:00) Monrovia, Reykjavik
To: admins; admins@tanu.com
Subject: Prod Cluster Yarn Utilization report
|
Report
generation time: |
2020-04-02
04:00:01.043615 |
|||
|
Total
Resource availabe: |
1440
Vcores, |
7.03TB |
||
|
Total
Resource Utilized(value): |
916
Vcores, |
3.95TB |
||
|
Total
Resource Utilized(%): |
63.61
%, |
56.24% |
||
|
Queue
Name |
Max
Allocation(cpu) |
Max
Allocation(memory) |
Cpu
Utilizied(%) |
Memory
Utilized(%) |
|
root.default |
1.0 |
1.0 |
0.0 |
0.0 |
|
root.dev1 |
1.0 |
1.0 |
0.0 |
0.0 |
|
root.hive |
1.0 |
1.0 |
0.0 |
0.0 |
|
root.qa1 |
60.0 |
60.0 |
23.0 |
23.0 |
|
root.admin1 |
1.0 |
1.0 |
0.0 |
0.0 |
|
root.dev2 |
1.0 |
1.0 |
0.0 |
0.0 |
|
root.bigdadmin |
1.0 |
1.0 |
0.0 |
0.0 |
|
root.qa2 |
1.0 |
1.0 |
0.0 |
0.0 |
|
root.cloudera |
1.0 |
1.0 |
0.0 |
0.0 |
|
root.bigdata |
1.0 |
1.0 |
0.0 |
0.0 |
|
root.reporting |
10.0 |
10.0 |
6.0 |
10.0 |
|
root.testing |
1.0 |
1.0 |
1.0 |
0.0 |
|
root.informatica |
1.0 |
1.0 |
0.0 |
0.0 |
|
root.admin2 |
1.0 |
1.0 |
0.0 |
0.0 |
|
root.banking |
5.0 |
5.0 |
5.0 |
4.0 |
|
root.informatica |
1.0 |
1.0 |
0.0 |
0.0 |
|
root.java |
1.0 |
1.0 |
0.0 |
1.0 |
|
root.users |
1.0 |
1.0 |
0.0 |
0.0 |