spark build path is cross-compiled with an incompatible version of Scala (2.10.0) - scala-ide

When i try to execute sparksql code in scala IDE im getting below error,Could anyone help me to sort out this please?
spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark Unknown Scala Version Problem
akka-remote_2.10-2.3.11.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark Unknown Scala Version Problem
akka-slf4j_2.10-2.3.11.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark Unknown Scala Version Problem
breeze_2.10-0.11.2.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark Unknown Scala Version Problem
breeze-macros_2.10-0.11.2.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark Unknown Scala Version Problem
chill_2.10-0.5.0.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark Unknown Scala Version Problem
jackson-module-scala_2.10-2.4.4.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark Unknown Scala Version Problem
json4s-ast_2.10-3.2.10.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark Unknown Scala Version Problem
json4s-core_2.10-3.2.10.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark Unknown Scala Version Problem
json4s-jackson_2.10-3.2.10.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark Unknown Scala Version Problem
quasiquotes_2.10-2.0.1.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark Unknown Scala Version Problem
scalatest_2.10-2.2.5.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark Unknown Scala Version Problem
spark-catalyst_2.10-1.4.0.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark Unknown Scala Version Problem
spark-core_2.10-1.5.2-tests.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark Unknown Scala Version Problem
spark-core_2.10-1.5.2.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark Unknown Scala Version Problem
spark-graphx_2.10-1.4.0.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark Unknown Scala Version Problem
spark-launcher_2.10-1.5.2.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark Unknown Scala Version Problem
spark-mllib_2.10-1.4.0-tests.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark Unknown Scala Version Problem
spark-mllib_2.10-1.4.0.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark Unknown Scala Version Problem
spark-network-common_2.10-1.5.2.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark Unknown Scala Version Problem
spark-network-shuffle_2.10-1.5.2.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark Unknown Scala Version Problem
spark-sql_2.10-1.4.0.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark Unknown Scala Version Problem
spark-streaming_2.10-1.5.2.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark Unknown Scala Version Problem
spark-unsafe_2.10-1.5.2.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark Unknown Scala Version Problem
spire_2.10-0.7.4.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark Unknown Scala Version Problem
spire-macros_2.10-0.7.4.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark Unknown Scala Version Problem
The version of scala library found in the build path of spark (2.10.4) is prior to the one provided by scala IDE (2.11.7). Setting a Scala Installation Choice to match. spark Unknown Scala Version Problem

In your project you are using jars built with different Scala versions; indeed from the log:
Scala IDE uses Scala 2.11.7
Apache Spark 1.5.2 built with Scala 2.10
You need to align the jars versions. Actually Spark 1.5.2 pre-build (downloadable from here) has built with Scala 2.10 for compatibility reasons (see here). On the web site there is a note:
Note: Scala 2.11 users should download the Spark source package and build with Scala 2.11 support.
In order to solve the issue, in your project use the same Scala version used to build Spark.
I suggest to try to switch to Scala 2.10 in your Scala IDE and it will solve the issue.
Eclipse + Maven
In my case, I'm using Eclipse with Scala IDE and Maven so I updated the Maven dependencies in this way:
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>2.10.6</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.10</artifactId>
<version>1.5.2</version>
</dependency>
Then, I changed the Scala version in the IDE: Right click on the project -> Scala -> set Scala installation or Right Click on the Scala Library Container -> Properties and choose the Scala 2.10 as shown below

Click right on your project, then select Properties.
Select Scala Compiler
Click "Use Project Setting"
Select Scala Installation select your scala version, for example, 2.11.11.

I had same problem. How I solved it.
Scala embedded Eclipse -> Project -> Right Click -> Project -> Scala Compiler -> Right side you will be getting list of Scala Installation-> Here choose Latest 2.10 bundle (dynamic).
I have tested and it worked fine for me.
Binod Suman

"-Xsource:2.11 -Ymacro-expand:none" solved my problem , just updated in right click project -> properties -> Scala Compiler , Additional Command Line Parameter , to fix this "build path is cross-compiled with an incompatible version of Scala (2.11.0). In case this report is mistaken, this check can be disabled in the compiler preference page."

Related

java.lang.NoSuchFieldError: defaultReader (JsonSmartJsonProvider.java:39)

I am using json-path-2.4.0 library in spark jobs which has a dependency on json-smart 2.x , but the spark jars default classpath folder (/usr/hdp/2.6.5.0-292/spark2/jars/) has json-smart 1.x which always gets precedence and I am unable to use the json-path 2.x library.
Facing the below error everytime I run :
java.lang.NoSuchFieldError: defaultReader
at com.jayway.jsonpath.spi.json.JsonSmartJsonProvider.(JsonSmartJsonProvider.java:39)
at com.jayway.jsonpath.internal.DefaultsImpl.jsonProvider(DefaultsImpl.java:21)
at com.jayway.jsonpath.Configuration.defaultConfiguration(Configuration.java:174)
Similar issue has been reported earlier :
JSON Path 2.3.0 conflicts with hadoop 2.7 Environment JSON-smart1.2.0.jar
But havent found any working solution. Please help.

Reference .netstandard 2.0 project in core 2.0 Runtime loading error when GenerateAssemblyInfo = false

When we add the reference, we are able to use the .netstandard libs classes in our .net core console project (in Visual Studio), and it compiles.
When we run the app, it immediately crashes with this error in the output window:
An unhandled exception of type 'System.IO.FileNotFoundException'
occurred in Unknown Module. Could not load file or assembly
'MyNetStandard20Assembly, Version=0.1.0.0, Culture=en-us,
PublicKeyToken=null'. The system cannot find the file specified.
Update / Cause (still looking for reason)
So we use AssemblyInfo files in our projects and in order to make these work with the new project structure we needed to add this to the first PropertyGroup in the .NET Standard project:
<GenerateAssemblyInfo>false</GenerateAssemblyInfo>
That then allows us to have a Properties/AssemblyInfo.cs file in the project, just like a .NET Framework project would allow.
Anyway, when we remove this from the .NET Standard project then the .NET Core project finds the assembly and is happy.
Does anyone know why?

Mysql version for Slick

I updated my Play app with Scala 2.12 and Play 2.6.1, also I've changed Slick to v3.2 . Before (Scala 2.11.11 and Play 2.5.14) everything was working fine with Mysql driver version 5.1.23, which is the officially supported version, as said in this page: https://github.com/slick/slick . Now I get
java.lang.ClassNotFoundException: slick.driver.MySQLDriver
Also after several sbt clean. Is there any form of dependence between mysql driver and scala?
It was obvious maybe, but I searched in the docs: http://slick.lightbend.com/doc/3.2.0/api/#slick.driver.package
and the driver package has changed from slick.driver to slick.jdbc.(db)Profile,
so now
slick.driver.MySQLDriver
became
slick.jdbc.MySQLProfile

Ceylon 1.2 Binary Compatibility

I tried to compile a source with Ceylon compiler version 1.2 that I previously compiled successfully with Ceylon compiler version 1.1 and I get the following error messages:
source/com/example/helloworld/module.ceylon:2: error: version '1.1.0' of module 'ceylon.net' was compiled by an incompatible version of the compiler (binary version 7.0 of module is not compatible with binary version 8.0 of this compiler)
import ceylon.net "1.1.0" ;
^
source/com/example/helloworld/module.ceylon:2: error: version '1.1.0' of module 'ceylon.collection' was compiled by an incompatible version of the compiler (binary version 7.0 of module is not compatible with binary version 8.0 of this compiler)
import ceylon.net "1.1.0" ;
^
source/com/example/helloworld/module.ceylon:2: error: version '1.1.0' of module 'ceylon.io' was compiled by an incompatible version of the compiler (binary version 7.0 of module is not compatible with binary version 8.0 of this compiler)
import ceylon.net "1.1.0" ;
^
source/com/example/helloworld/module.ceylon:2: error: version '1.1.0' of module 'ceylon.file' was compiled by an incompatible version of the compiler (binary version 7.0 of module is not compatible with binary version 8.0 of this compiler)
import ceylon.net "1.1.0" ;
I suppose that " ... binary version 8.0 ... " in the error message refers to the Java version.
In both attempts to compile (first with Ceylon 1.1, second with 1.2) I used Java version 8 and I dont want to change that back to 7.
Does it help to compile the Ceylon SDK with Java version 8? How can I do that separately from the entire Ceylon distribution?
How can I import the sources of Ceylon SDK into my project and compile it together with my project?
The binary versions in the error message refer to the Ceylon binary versions, which I guess by unfortunate coincidence, happen to match current JVM versions.
Ceylon is compatible with both JVM 7 and JVM 8, but Ceylon 1.2.0 programs must use Ceylon 1.2.0 modules; binary compatibility with Ceylon 1.1.0 was not maintained.
The solution here is to simply change the import to import ceylon.net "1.2.0";.
No, that’s actually the binary version of Ceylon, unrelated to Java. Ceylon 1.1 was binary version 7, and Ceylon 1.2 is binary version 8. Unfortunately, we weren’t able to provide binary compatibility between these releases.
You’ll have to use the 1.2.0 SDK modules with Ceylon 1.2.

How to add javax.swing.SwingUtilities dependency to SBT?

I'm starting to develop a Scala application with Swing using SBT. I figured out that I need two dependencies for a start, so that's scala-swing and javax.swing.SwingUtilities.
I've been searching the web, maven repositories and github, but stil couldn't find, where did the javax.swing package go.
So far I have found javax in Maven Repos, but javax.swing is not listed there for some reason.
I tried to add a javax dependency to my Build.scala:
val javax = "javax" % "javaee-api" % "7.0"
SBT downloaded several packages. Then I launched the terminal:
scala> import javax.swing.SwingUtilities
import javax.swing.SwingUtilities
scala> SwingUtilities.invokeLater()
<console>:9: error: not enough arguments for method invokeLater: (x$1: Runnable)Unit.
Unspecified value parameter x$1.
SwingUtilities.invokeLater()
^
That's scala console being launched from sbt project. So as you can see, import was successful and the console knows about invokeLater() method. But IntelliJ Idea still does not, it marks javax.swing as unresolvable, though it has downloaded the packages.
I'm completely stuck here.
javax.swing is part of any standard Java SE installation (see http://docs.oracle.com/javase/6/docs/api/ - there you have SwingUtilities), so there is no need to add a dependency. You only need the scala-swing dependency:
libraryDependencies += "org.scala-lang" % "scala-swing" % scalaVersion.value
If IntelliJ IDEA doesn't see javax.swing, you have probably not yet defined an "SDK". Go to File -> Project Structure -> Platform Settings -> SDKs. There you should have at least one entry such as "1.6" for Java 1.6 or "1.7" for Java 1.7. If not, press the "+" to add one and locate the appropriate Java home directory for the version you want to use (depends on your OS).