-->
  • Recent Articles

    How to Configure the Spark SQL Hive Warehouse Directory

    How to Configure the Spark SQL Hive Warehouse Directory

    Hive

    Since Spark 2.0.0, Spark references "spark.sql.warehouse.dir" as the default Spark SQL Hive Warehouse location. To create databases, both the "spark-defaults.conf" and "spark-thrift-sparkconf.conf" files should have a spark.sql.warehouse.dir property that has the same value as the hive.metastore.warehouse.dir property. The default value of this property is usually /apps/hive/warehouse:


    spark.sql.warehouse.dir=/apps/hive/warehouse

    On an Ambari cluster, select Spark2 > Configs, then use "Add Property" in "Custom spark2-defaults" and "Custom spark2-thrift-sparkconf" to add a spark.sql.warehouse.dir property with the value /apps/hive/warehouse. you can change this value to your desired value.


    HIVE


    Now Click on "Add Property" and add your new value. Then Click on Save.



    No comments