SQL_to_HDFS ( With out Using Sqoop)




scala> val data=sqlContext.load("jdbc",Map("url"->"jdbc:mysql://localhost/gnr?user=root&password=cloudera","dbtable"->"emp"))



jdbcDF.write
  .format("jdbc")
  .option("url", "jdbc:postgresql:dbserver")
  .option("dbtable", "schema.tablename")
  .option("user", "username")
  .option("password", "password")

  .save()


warning: there were 1 deprecation warning(s); re-run with -deprecation for details
data: org.apache.spark.sql.DataFrame = [id: int, name: string, sal: int]

scala> data.show()
+---+----+----+
| id|name| sal|
+---+----+----+
| 10| aaa|5000|
| 20|bbbb|8222|
| 30| ccc|4500|
+---+----+----+


scala> data.map(x=>x.mkString(",")).saveAsTextFile("gnremp")

----------
df.write
  .format("jdbc")
  .option("url", "jdbc:postgresql:dbserver")
  .option("dbtable", "schema.tablename")
  .option("user", "username")
  .option("password", "password")

  .save()


Comments