-->
  • Recent Articles

    python connect impala

    python connect impala


    hive


    To use python to connect to hive, first download the following packages

    pip install pure-sasl
    pip install thrift
    pip install thrift-sasl
    pip install cython
    pip install impyla

    If there are any installation error try updating setuptools.

    Update setuptools

    pip install --upgrade setuptools


    Test  the code  as follows

    from impala.dbapi import connect
    conn = connect(host='10.0.12.160',port=10000,auth_mechanism="PLAIN")
    cur = conn.cursor()
    cur.execute("select * from dice_raw.card limit 10")
    for row in cur:
        print(row)

    If you report an error like below , then Use thrift 0.9.3 version

    Traceback (most recent call last):
      File "updateImpala.py", line 65, in <module>
        main(sys.argv[1:])
      File "updateImpala.py", line 48, in main
        cur = conn.cursor()
      File "/Library/Python/2.7/site-packages/impala/hiveserver2.py", line 125, in cursor
        session = self.service.open_session(user, configuration)
      File "/Library/Python/2.7/site-packages/impala/hiveserver2.py", line 1016, in open_session
        resp = self._rpc('OpenSession', req)
      File "/Library/Python/2.7/site-packages/impala/hiveserver2.py", line 944, in _rpc
        response = self._execute(func_name, request)
      File "/Library/Python/2.7/site-packages/impala/hiveserver2.py", line 961, in _execute
        return func(request)
      File "/Library/Python/2.7/site-packages/impala/_thrift_gen/TCLIService/TCLIService.py", line 175, in OpenSession
        self.send_OpenSession(req)
      File "/Library/Python/2.7/site-packages/impala/_thrift_gen/TCLIService/TCLIService.py", line 182, in send_OpenSession
        args.write(self._oprot)
      File "/Library/Python/2.7/site-packages/impala/_thrift_gen/TCLIService/TCLIService.py", line 1250, in write
        oprot.trans.write(fastbinary.encode_binary(self, (self.__class__, self.thrift_spec)))
    TypeError: expecting list of size 2 for struct args


    Use thrift 0.9.3 version

    pip install thrift==0.9.3


    No comments