sbt - Spark 1.2 SQL code does not work with Spark 1.3 SQL code -

I have used this build.sbt in the local package directory so far

  names : = "spark27_02" version: = "1.0" scalaVersion: = "2.10.4" sbtVersion: = "0.13.7" libraryDependencies + = "org.apache.spark" %% "Spark core"% "1.2.1" libraryDependencies + = "Orgkapachekspark" %% "spark-streaming"% "1.2.1" Laibreridispensians + "" org.apache.spark "%%" Spark-SQL "%" 1.2.1 "Laibrerivr dependence + =" Organization .apache.hadoop "%" Thup-HDFS "%" 2.5.0 " 

I just wanted to use version 1.3.0, so I The 1.3.0 version of all package compiled Spark, but is not SparkSQL, so I used to use MavenCentral to check that

  libraryDependencies = "org.apache.spark"% "Spark sql_2.10"% "1.3.0"  

but still is not working. I am updating SBT from SBT Shell using BDW 2.10.4 using Scala

Any silly thing I am doing wrong?

Any help is appreciated.

  name: = "Marzia2" version: = "1.0" scalaVersion: = "2.10.4" sbtVersion:  
 Example with this build on SPARC webpage. "0.13.7" libraryDependencies + = "org.apache. spark"% "spark-Kor_2kl0"% "1.3.0" Laibreridiprensijh + + "org.apache.spark"% "spark-Stryming_2kl0"% "1.3 .0 "LibraryDiscense +" "org.apache.spark"% "Spark-sql_2.10"% "1.3.0"  

do

  sbt package  

i get

  [information /home/cloudera/IdeaProjects/Marzia2/target/scala-2.10/classes to] ... compilation 1 Scala Source [error] / home / cloudera / IdeaProjects / Marzia2 / src / m Ky / scala / prova_sql.scala: 35: Price createSchemaRDD org.apache.spark.sql.SQLContext [error] Import sqlContext.createSchemaRDD [error] ^ [error] / home / Cloudera / IdeaProjects / Marzia2 is not a member of the / src / main / scala / prova_sql.scala38: value registerTempTable two errors found org.apache.spark.rdd.RDD [prova_sql.Person] [eRROR] people.registerTempTable ( "people") [error] ^ [error] is not a member of [ Error] (compilation: compilation) compilation failed  

And if I use new features like implicits to define spark references, then I There is no error in the sparkscull error.

The problem is that the schema RDD became a dataframe . In fact, you should use

  import sqlContext._  

instead of a specific import, because it will give you future proof against unbalanced changes , But if you really want, you can use

  import sqlContext.implicits  

, but the second part is that 1.3.0 Compatibility broke and now it has been stopped from the API perspective, then you need to do the following:

• implicits poo They are not blown in the form of 1.2. To use them, you now have to: rdd.toDF (). RegisterTempTable ("xyz")

Note toDF

Now when the API is locked, I have more intuitive Can not think of a way to add built-in You have conflicting built-in <

>

The underlying conversion now only raises the RDD which is made from the product (i.e., case classes or tuples) Instead of applying with DDF happens.