Python CGI Webserver with SSL

This is sample code to run python CGI Web server with SSL. Note that i used CA signed certificate  due to that i need to provide CA certificate as well in my python code which may not be necessary if you use self signed certificate.

       
#!/usr/bin/python

import BaseHTTPServer, SimpleHTTPServer,CGIHTTPServer

import ssl

import cgitb; cgitb.enable()



#httpd = BaseHTTPServer.HTTPServer(('web.tanu.com', 8000), SimpleHTTPServer.SimpleHTTPRequestHandler)

handler = CGIHTTPServer.CGIHTTPRequestHandler

httpd = BaseHTTPServer.HTTPServer(('web.tanu.com', 8000), handler)

handler.cgi_directories = ["/cgi-bin"]



httpd.socket = ssl.wrap_socket (httpd.socket,keyfile='./certs/key1.pem', certfile='./certs/cert.pem',ca_certs='./ca_cert.pem,server_side=True,do_handshake_on_connect=True)

handler.have_fork=False

httpd.serve_forever()



       
 

How to implement SPNEGO authentication in kerberos with python

Let me explain my best  how spnego authentication work and how can we implement SPNEGO authentication in a few lines in python.

In this test, I used MIT kerberos server (Active directory installation and configuration is more painful for me) refer my blog on how to setup and configure MIT kerberos in centos. Linux Ldap authentication with kerberos backend and openldap SASL Passthrough  and kerberos  (python module to implement the spengo authentication (pip install kerberos))


We need to create service principle in Kerberos server for the domain we are going to use www.sathish.com. since webserver use HTTP protocol, kerberos service principle would be HTTP.

So we need to create HTTP service principle with below name for www.sathish.com. ( TANU.COM is my kerberos domain.)

HTTP/www.sathish.com@TANU.COM

Run following commands in kerberos server to create the keytab.

kadmin.local -q "addprinc -randkey HTTP/www.sathish.com@TANU.COM"
kadmin.local -q "ktadd -k /www/http/secure/www.sathish.com.keytab HTTP/www.sathish.com

set below environment variable to point the keytab

export KRB5_KTNAME=/www/http/secure/www.sathish.com.keytab

(if we dont set above variable python code with throw below exception

    rc, state = kerberos.authGSSServerInit('HTTP')
GSSError: (('Unspecified GSS failure.  Minor code may provide more information', 851968), ('', 100002))

)

Now we are good to run our own code to implement the spnego kerberos authentication.


Webserver (simple python http server with custom code) running on CENTOS  with url http:/www.sathish.com



Client machine(one Linux  and  one windows 10 desktop)

How SPNEGO work:

1) client request url using Internet explore   ----> http://www.sathish.com

2) webserver get request and check  "Authorization" header. if header not present in the request , webserver send the response with 401 and WWW-Authenticate','Negotiate' header  back to client

                       header=s.headers.get('Authorization')
                        if not header:
                        s.send_response(401)
                        s.send_header('WWW-Authenticate','Negotiate')

3) Internet explore get the response from webserver and request URL with Authorization header with valid token.

4)  Then webserver read the token from Authorization header and decrypt  token using service key stored in the keytab for this HTTP Service principle HTTP/www.sathish.com@TANU.COM.

If the decryption is successful then user token is valid and we can mark user authentication is completed.

               rc, state = kerberos.authGSSServerInit('HTTP')
                if rc != kerberos.AUTH_GSS_COMPLETE:
                        return None
                rc = kerberos.authGSSServerStep(state, token)
                if rc == kerberos.AUTH_GSS_COMPLETE:
                        user = kerberos.authGSSServerUserName(state)

  if token is valid, webserver can  retrieve the username.

5) Post Authentication webserver response back to client.

Here is the full code for testing

       
import BaseHTTPServer
from SimpleHTTPServer import SimpleHTTPRequestHandler
import sys
import kerberos
import re

class AuthHandler(SimpleHTTPRequestHandler):
    ''' Main class to present webpages and authentication. '''
    def do_HEAD(self):
        print "send header"
        self.send_response(200)
        self.send_header('Content-type', 'text/html')
        self.end_headers()

    def do_AUTHHEAD(self):
        print "send header"
        self.send_response(401)
        #self.send_header('WWW-Authenticate', 'Basic realm=\"Test\"')
        self.send_header('WWW-Authenticate', 'Negotiate')
        self.send_header('Content-type', 'text/html')
        self.end_headers()

    def do_GET(self):
        ''' Present frontpage with user authentication. '''
        if self.headers.getheader('Authorization') == None:
            self.do_AUTHHEAD()
            self.wfile.write('no auth header received')
            pass
        #elif self.headers.getheader('Authorization') == 'Negotiate':
        elif re.match(r'^Negotiate', self.headers.getheader('Authorization')):
            #print self.headers.getheader('Authorization')
            header=self.headers.getheader('Authorization')
            token = ''.join(header.split()[1:])
            print token
            rc, state = kerberos.authGSSServerInit('HTTP')
            if rc != kerberos.AUTH_GSS_COMPLETE:
                return None
            rc = kerberos.authGSSServerStep(state, token)
            if rc == kerberos.AUTH_GSS_COMPLETE:
                user = kerberos.authGSSServerUserName(state)
                print user
                SimpleHTTPRequestHandler.do_GET(self)
                pass
        else:
            self.do_AUTHHEAD()
            self.wfile.write(self.headers.getheader('Authorization'))
            self.wfile.write('not authenticated')
            pass

def test(HandlerClass = AuthHandler,
         ServerClass = BaseHTTPServer.HTTPServer):
    BaseHTTPServer.test(HandlerClass, ServerClass)


if __name__ == '__main__':
    if len(sys.argv)<2: code="" port="" print="" simpleauthserver.py="" sys.exit="" test="" usage="">
 

Here is the curl with negotiate request output



[root@krbserver ~]# curl --negotiate -u : -b ~/cookiejar.txt -c ~/cookiejar.txt  http://www.sathish.com/ -v
* About to connect() to www.sathish.com port 80 (#0)
*   Trying 192.168.100.31...
* Connected to www.sathish.com (192.168.100.31) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: www.sathish.com
> Accept: */*
>
* HTTP 1.0, assume close after body
< HTTP/1.0 401 Unauthorized
< Server: BaseHTTP/0.3 Python/2.7.5
< Date: Sun, 13 May 2018 17:09:17 GMT
< WWW-Authenticate: Negotiate
* Closing connection 0
* Issue another request to this URL: 'http://www.sathish.com/'
* About to connect() to www.sathish.com port 80 (#1)
*   Trying 192.168.100.31...
* Connected to www.sathish.com (192.168.100.31) port 80 (#1)
* Server auth using GSS-Negotiate with user ''
> GET / HTTP/1.0
> Authorization: Negotiate YIICWwYJKoZIhvcSAQICAQBuggJKMIICRqADAgEFoQMCAQ6iBwMFACAAAACjggFiYYIBXjCCAVqgAwIBBaEKGwhUQU5VLkNPTaIZMBegAwIBA6EQMA4bBEhUVFAbBmtyYmNuMaOCASowggEmoAMCARChAwIBA6KCARgEggEUoewKZoNfchhCOjox2e4y465Wnz2E94henxLTzQq70A270SbVPef5oIkqXXupDK9/JPhT2QPRUXEhWYoH41KwxfK4Q28H2OLpzkvCCk0wVLjNFrJZjhEm1mxV4eKUl4AAi/nSwYUCZGAEFrEUVwWQwP8apFCsGolvZXP9Skc0XViAZTM5JKL6mVgQcwwuvs+J6Cf6elzZPR+7BLrJRFHlUTvMS3C8H+WClAh9KDeSvSbxYvG4ZRhNgXqRrD259O5iGKySf126ToG00Zece4Fz0rHjSRJUnrV4H2Npku2sMSo7M9DnwBiDW8kQ84yQnpXbc1hOqiFIOWRc31ZuQ2O0OUormHm3mUZY+ABC2zA8BdIPLZvspIHKMIHHoAMCARCigb8EgbxC+6wp6oKhdTtVS6nFomRylJ1p8t1hn8nqs/rsNTHoia6csinnPP2ZR2oqydnaVq9cGMqA6q7U5Yz1iXlf0dGOWmOewucN8URLkYDC6sne2acqNkWCpOVyJJ9/AaC5QTFvVyR+Nj7aRW3LpSgJdrwrMybrfgYawthj0q44z7+FQi1T0K42sLmTvelkqvXFdVBn2/aqcaXCkkA9OI5IrZH0fqwt25nMm/mTkhsgH+ntuzjtm4VDViLV2DuZcA==
> User-Agent: curl/7.29.0
> Host: www.sathish.com
> Accept: */*
>
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Server: BaseHTTP/0.3 Python/2.7.5
< Date: Sun, 13 May 2018 17:09:19 GMT
< Content-type: text/html
<
* Closing connection 1
<html><head><title>Title goes here.</title></head><body><p>This is a test.</p></body></html>[root@krbserver ~]#

User defined exception class in python

Today i learned something about user defined exception class in python and posting  in my blog so that i can remember in future :)


This is my check_age.py file

from own_Exeption import teenageException,kidException,childException

def age(x):
        try:
                if( x >= 14 ) and (x <= 19):
                        raise teenageException
                elif(x >= 8 ) and (x <= 13) :
                        raise kidException
                elif(x >= 2 ) and (x <= 7) :
                        raise childException
                #else:
                #       print("Ags is : "+str(x))
        except teenageException:
                print("WARNING : You are teenage not allowed here")
        except kidException:
                print("WARNING: you are kid not allowed here")
        except childException:
                print("WARNING: you are child not allowed here")
        else:
                print("INFO: You are the right age {} adult person".format(x))
        finally:
                print("INFO: End of the try/except block")

age(3)
age(17)
age(21)

Here is my user defined exception class

cat own_Exeption.py

class teenageException(Exception):
        pass
class kidException(Exception):
        pass
class childException(Exception):
        pass  

Initially i tried without try/except block. it seems like execution getting halt as soon as raise block invoked. So i tried with try/except block to catch the respective and print the warning message. here is my output.

       

WARNING: you are child not allowed here
INFO: End of the try/except block
WARNING : You are teenage not allowed here
INFO: End of the try/except block
INFO: You are the right age 21 adult person
INFO: End of the try/except block


       
 


i could also use raise statement inside except block like below to halt the code from further execution

       

from own_Exeption import teenageException,kidException,childException

def age(x):
        try:
                if( x >= 14 ) and (x <= 19):
                        raise teenageException
                elif(x >= 8 ) and (x <= 13) :
                        raise kidException
                elif(x >= 2 ) and (x <= 7) :
                        raise childException
                #else:
                #       print("Ags is : "+str(x))
        except teenageException:
                print("WARNING : You are teenage not allowed here")
                raise teenageException
        except kidException:
                print("WARNING: you are kid not allowed here")
                raise kidException
        except childException:
                print("WARNING: you are child not allowed here")
                raise childException
        else:
                print("INFO: You are the right age {} adult person".format(x))
        finally:
                print("INFO: End of the try/except block")

age(3)
age(17)
age(21)


       
 

This  will stop execution  withing first error.

       

WARNING: you are child not allowed here
INFO: End of the try/except block
Traceback (most recent call last):
  File "sat1.py", line 27, in 
    age(3)
  File "sat1.py", line 21, in age
    raise childException
own_Exeption.childException

       
 

Linux Ldap authentication with kerberos backend and openldap SASL Passthrough

Installation and configuration of Openldap|Kerberos server with openldap SASL Passthrough

This is the article for setting up Centos7 with openldap authentication and kerberos backend to enable SSH single sign on(SSO). And SASL service will be installed and configured to allow sasl pass-through on openldap.

What is Openldap SASL  pass-through

 To explain in simple words, Openldap SASL pass-through delegating authentication operation from openldap to kerberos(In our case). So that we can maintain just one password for both ldap and kerberos user. otherwise we have to use one password for ldap authentication another one for kerberos authentication and ticket generation.

All we need to do is, while creating user in ldap using ldif file we have to specify below value instead of user passoword.

userPassword: {SASL}testuser1@TANU.COM

SASL Service installation:

My shell script will take care of installation and configuration of sasl service as part of ldap/kerberos server setup by invoking below function.

install_sasl_service(){
	yum -y install cyrus-sasl
	banner_msg "INFO: Creating /etc/sasl2/slapd.conf file for LDAP Sasl authencation"
	banner_msg "INFO: dont fotget to copy /etc/ssl/certs/${kerberos_server_hostname}/MyRootCA.pem file from kerber server to all the client in same folder path otherwise LDAP bind will not work"
	cat > /etc/sasl2/slapd.conf <<- "EOF"
	mech_list: external gssapi plain
	pwcheck_method: saslauthd
	EOF
	echo "SOCKETDIR=/var/run/saslauthd" >>/etc/sysconfig/saslauthd
	echo "MECH=kerberos5" >>/etc/sysconfig/saslauthd
	echo "KRB5_KTNAME=/etc/krb5.keytab" >>/etc/sysconfig/saslauthd
	systemctl restart saslauthd.service
	}

Openldap|kerberos server installation:

I have used two google cloud VM instances for setting up Openldap and kerberos server and ldap client.

Setting up kerberos and openldap server

Kerberos-server detail --> Centos7 --> hostname is idm.tanu.com

login into the kerberos-server host.

Install git client - yum install -y install git-core

Download the Installation scripts -  git clone https://github.com/skumarx87/openldap_MIT-Kerberos_installation


Edit the shell script and update the main function with your details:

In my tutorial i have used kerberos domain with TANU.COM everywhere.

and then run the script with following option for server setup:

cd openldap_MIT-Kerberos_installation

./kerberos_ldap_installation.sh server_setup

This shell script with server installation input  will invoke following functions. you find the list of commands being executed by each function.

                create_root_ca_pair
                creating_ldap_ssl_pair_pem
                openldap_installation
                install_sasl_service
                enable_ldap_tls
                install_kerberos_server
                enable_kerberos_ldap_backend
                creating_kerberos_db



Creating test users and groups

I primarily developed this script for to setup hadoop cluster for that set of users and groups need to be created. we can create same users and groups to test our installation.

Below file has users and respective groups information which will be created in Ldap and kerberos with default password mentioned in the script hadoop_users_password="support123"

[root@idm openldap_MIT-Kerberos_installation]# cat hadoop_users_map.txt
##############################
#username:group1,group2,etc  #
##############################
cloudera-scm:cloudera-scm
accumulo:accumulo
flume:flume
hbase:hbase
hdfs:hdfs,hadoop
hive:hive
httpfs:httpfs
hue:hue
apache:apache
impala:impala,hive
kafka:kafka
kms:kms
keytrustee:keytrustee
kudu:kudu
llama:llama
mapred:mapred,hadoop
oozie:oozie
solr:solr
spark:spark
sentry:sentry
sqoop:sqoop
sqoop2:sqoop,sqoop2
yarn:yarn,hadoop

you will see below logs while adding.

adding new entry "uid=sqoop2,ou=People,dc=tanu,dc=com"

adding new entry "uid=yarn,ou=People,dc=tanu,dc=com"

adding new entry "uid=zookeeper,ou=People,dc=tanu,dc=com"

Principal "sqoop2@TANU.COM" created.
Creating kdc user principle yarn
Authenticating as principal root/admin@TANU.COM with password.
WARNING: no policy specified for yarn@TANU.COM; defaulting to no policy
Principal "yarn@TANU.COM" created.
Creating kdc user principle zookeeper
Authenticating as principal root/admin@TANU.COM with password.
WARNING: no policy specified for zookeeper@TANU.COM; defaulting to no policy
Principal "zookeeper@TANU.COM" created.

Setting up kerberos and openldap client:

run this command on all the client hosts(we could run on kerberos server  host as well)

./kerberos_ldap_installation.sh client_setup

---------------------------------------------------
INFO: creating krb5.conf file                                         
---------------------------------------------------
Cannot set persistent booleans without managed policy.
---------------------------------------------------
INFO: Creating host keytab file                                       
---------------------------------------------------
Authenticating as principal root/admin@TANU.COM with password.
WARNING: no policy specified for user5@TANU.COM; defaulting to no policy
Principal "user5@TANU.COM" created.
Authenticating as principal root/admin@TANU.COM with password.
WARNING: no policy specified for host/idm.tanu.com@TANU.COM; defaulting to no policy
Principal "host/idm.tanu.com@TANU.COM" created.
Authenticating as principal root/admin@TANU.COM with password.
Entry for principal host/idm.tanu.com with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/etc/krb5.keytab.
Entry for principal host/idm.tanu.com with kvno 2, encryption type aes---------------------------------------------------
INFO: creating krb5.conf file                                         
---------------------------------------------------
Cannot set persistent booleans without managed policy.
---------------------------------------------------
INFO: Creating host keytab file                                       
---------------------------------------------------
Authenticating as principal root/admin@TANU.COM with password.
WARNING: no policy specified for user5@TANU.COM; defaulting to no policy
Principal "user5@TANU.COM" created.
Authenticating as principal root/admin@TANU.COM with password.
WARNING: no policy specified for host/idm.tanu.com@TANU.COM; defaulting to no policy
Principal "host/idm.tanu.com@TANU.COM" created.
Authenticating as principal root/admin@TANU.COM with password.
Entry for principal host/idm.tanu.com with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/etc/krb5.keytab.
Entry for principal host/idm.tanu.com with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/etc/krb5.keytab.128-cts-hmac-sha1-96 added to keytab WRFILE:/etc/krb5.keytab.


Verify the server installation

[root@idm openldap_MIT-Kerberos_installation]# kadmin.local
Authenticating as principal root/admin@TANU.COM with password.
kadmin.local:  listprincs
K/M@TANU.COM
krbtgt/TANU.COM@TANU.COM
kadmin/idm.tanu.com@TANU.COM
kadmin/admin@TANU.COM
kadmin/changepw@TANU.COM
kiprop/idm.tanu.com@TANU.COM
root/admin@TANU.COM

verify client installation:

login into client host with one of the hadoop user hdfs and password in default support123.  you should able to login

[skumarx87@idm ~]$ su - hdfs
Password:
Last login: Sun Sep  1 02:32:20 UTC 2019 on pts/0

[hdfs@idm ~]$ id

uid=3005(hdfs) gid=2005(hdfs) groups=2005(hdfs),2028(hadoop) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

[hdfs@idm ~]$ 

There shouldn't be any kerberos ticket by default. we confirm by running klist. now try to create kerberos ticket by kinit command with same password.

[hdfs@idm ~]$ klist
klist: No credentials cache found (filename: /tmp/krb5cc_3005)
[hdfs@idm ~]$ kinit hdfs
Password for hdfs@TANU.COM:
[hdfs@idm ~]$ klist
Ticket cache: FILE:/tmp/krb5cc_3005
Default principal: hdfs@TANU.COM

Valid starting       Expires              Service principal
09/01/2019 02:34:18  09/02/2019 02:34:18  krbtgt/TANU.COM@TANU.COM
[hdfs@idm ~]$


issues and resolutions

Trying to run kadmin from the client machine since kadmin.local command will work only from the kerberos server.

Error:
kadmin: Client 'root/admin@TANU.COM' not found in Kerberos database while initializing kadmin interface

Solution :

I forgot to create root principle in kerberos server. after i ran below command, i was able to run kadmin command from any client hosts

kadmin.local -q "addprinc -pw ${KDC_ADMIN_PASSWD} root/admin@${KRB_DOMAIN_NAME}"


Error:
saslauthd[3123]: auth_krb5: k5support_verify_tgt

Solution :

kadmin.local -q "addprinc -randkey host/${CLIENT_FQDN_HOST}@TANU.COM"
kadmin.local -q "ktadd -k /etc/krb5.keytab host/${CLIENT_FQDN_HOST}

cat /etc/sysconfig/saslauthd

KRB5_KTNAME=/etc/krb5.keytab