Unable to authenticate with kerberos for sentry

Description

I am trying to use sentry for cdh 5.12, kylo 0.8.3. I have kerberos enabled in the environment(tested, Ingestion feed is working). For sentry configuration, i have followed your documentation and here is my authorization.sentry.properties file:
beeline.connection.url=jdbc:hive2://localhost:10000/default;principal=hive/containerhadoop.container.com@CLOUDERA
beeline.drive.name=org.apache.hive.jdbc.HiveDriver
beeline.userName=nifi
beeline.password=
hdfs.hadoop.configuration=/etc/hadoop/conf/hdfs-site.xml,/etc/hadoop/conf/core-site.xml
authorization.sentry.groups=sentryAdmin,sentryUser
sentry.kerberos.principal=nifi
sentry.kerberos.KeytabLocation=/etc/nifi.headless.keytab
sentry.IsKerberosEnabled=true
authorization.sentry.type=static
authorization.sentry.groups=sentryAdmin,sentryUser
authorization.sentry.unix.group.filePath=/etc/group
authorization.sentry.ldap.url=ldap://192.168.56.105:389
authorization.sentry.ldap.authDn=cn=shashi,dc=teradata,dc=com
authorization.sentry.ldap.password=thinkbig123
authorization.sentry.ldap.authenticator.groupDnPatterns=dc=teradata,dc=com

Here is the stacktrace of error:
ERROR shared-1:SentryAuthorizationService:535 - Unable to authenticate with Kerberos while creating Sentry Policy Login failure for nifi from keytab /etc/nifi.headless.keytab
2017-12-18 12:13:21 ERROR shared-1:SentryAuthorizationService:187 - Error Creating Sentry HDFS Policy using Kerberos Authenticationjava.io.IOException: Login failure for nifi from keytab /etc/nifi.headless.keytab
2017-12-18 12:13:21 ERROR shared-1:BaseHadoopAuthorizationService:103 - Error creating Kylo Authorization policy after metadata property change event
java.lang.RuntimeException: java.lang.RuntimeException: java.io.IOException: Login failure for nifi from keytab /etc/nifi.headless.keytab
at com.thinkbiganalytics.datalake.authorization.SentryAuthorizationService.createOrUpdateReadOnlyHdfsPolicy(SentryAuthorizationService.java:188)
at com.thinkbiganalytics.datalake.authorization.service.BaseHadoopAuthorizationService$FeedPropertyChangeDispatcher.notify(BaseHadoopAuthorizationService.java:86)
at com.thinkbiganalytics.datalake.authorization.service.BaseHadoopAuthorizationService$FeedPropertyChangeDispatcher.notify(BaseHadoopAuthorizationService.java:72)
at com.thinkbiganalytics.metadata.event.reactor.ReactorMetadataEventService$ListenerConsumer.accept(ReactorMetadataEventService.java:136)
at com.thinkbiganalytics.metadata.event.reactor.ReactorMetadataEventService$ListenerConsumer.accept(ReactorMetadataEventService.java:125)
at reactor.bus.EventBus$3.accept(EventBus.java:317)
at reactor.bus.EventBus$3.accept(EventBus.java:310)
at reactor.bus.routing.ConsumerFilteringRouter.route(ConsumerFilteringRouter.java:72)
at reactor.bus.EventBus.accept(EventBus.java:591)
at reactor.bus.EventBus.accept(EventBus.java:63)
at reactor.core.dispatch.AbstractLifecycleDispatcher.route(AbstractLifecycleDispatcher.java:160)
at reactor.core.dispatch.SingleThreadDispatcher$SingleThreadTask.run(SingleThreadDispatcher.java:79)
at reactor.core.dispatch.RingBufferDispatcher$3.onEvent(RingBufferDispatcher.java:156)
at reactor.core.dispatch.RingBufferDispatcher$3.onEvent(RingBufferDispatcher.java:153)
at reactor.jarjar.com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: java.io.IOException: Login failure for nifi from keytab /etc/nifi.headless.keytab
at com.thinkbiganalytics.datalake.authorization.SentryAuthorizationService.authenticatePolicyCreatorWithKerberos(SentryAuthorizationService.java:536)
at com.thinkbiganalytics.datalake.authorization.SentryAuthorizationService.createOrUpdateReadOnlyHdfsPolicy(SentryAuthorizationService.java:174)
... 17 more
Caused by: java.io.IOException: Login failure for nifi from keytab /etc/nifi.headless.keytab
at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1145)
at com.thinkbiganalytics.kerberos.KerberosTicketGenerator.generateKerberosTicket(KerberosTicketGenerator.java:52)
at com.thinkbiganalytics.datalake.authorization.SentryAuthorizationService.authenticatePolicyCreatorWithKerberos(SentryAuthorizationService.java:531)
... 18 more
Caused by: javax.security.auth.login.LoginException: Unable to obtain password from user

Environment

None

Activity

Show:
Ankita Makwana
January 2, 2018, 3:17 AM

I have tested kerberos test client and it ran successfully, but still that issue is there. Output of kerberos test client is :

[root@containerkylo bin]# ./java -jar /kylo-kerberos-test-client-0.8.4.jar

Which environment are you in? Enter 1 for HDP or 2 for Cloudera: 2

Hit enter to default to: /etc/hadoop/conf/core-site.xml,/etc/hadoop/conf/hdfs-site.xml,/usr/hdp/current/hive-client/conf/hive-site.xml
Please enter the list of configuration resources:

Hit enter to default to: /etc/security/keytabs/hive-thinkbig.headless.keytab
Please enter the keytab file location: /etc/kylo.headless.keytab

Hit enter to default to: hive/sandbox.hortonworks.com@sandbox.hortonworks.com
Please enter the real user principal name: kylo

Please enter Y/N (default is N)
Do you want to test with a proxy user: N

Hit enter to default to: hdfs://sandbox.hortonworks.com:8020
Please enter the HDFS URL: hdfs://containerhadoop.container.com:8020

Hit enter to default to: jdbc:hive2://localhost:10000/default
Please enter the Hive base connection string: jdbc:hive2://containerhadoop.container.com:10000/default;principal=hive/containerhadoop.container.com

Executing Kinit to generate a kerberos ticket
No Proxy User
log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Generating Kerberos ticket for principal: kylo@CLOUDERA at key tab location: /etc/kylo.headless.keytab

Sucessfully got a kerberos ticket in the JVM
current user is: kylo@CLOUDERA
File Count: 9
Generating Kerberos ticket for principal: kylo@CLOUDERA at key tab location: /etc/kylo.headless.keytab

Sucessfully got a kerberos ticket in the JVM
Hive URL: jdbc:hive2://containerhadoop.container.com:10000/default;principal=hive/containerhadoop.container.com
creating statement

Executing the Hive Query:

List of Databases
default
...................................................
Tested with nifi principal also. it worked fine.

Assignee

Unassigned

Reporter

Ankita Makwana

Labels

None

Reviewer

None

Story point estimate

None

Components

Priority

High
Configure