Testing a DSPComplex ROM - chisel

I'm working on building a DSPComplex ROM still and have hit what I think may be an actual Chisel problem.
I've built the ROM, can generate a verilog output from the code that looks reasonable, but can't seem to test the module with even the most basic of testers. I've simplified it below to the most basic checking.
The error is a stack overflow like the following:
$ sbt 'testOnly taylor.TaylorTest'
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=8G; support was removed in 8.0
[info] Loading settings from plugins.sbt ...
[info] Loading project definition from /home/jcondley/Zendar/kodo/ZenFPGA/chisel/project
[info] Loading settings from build.sbt ...
[info] Set current project to zen-chisel (in build file:/home/jcondley/Zendar/kodo/ZenFPGA/chisel/)
[info] Compiling 1 Scala source to /home/jcondley/Zendar/kodo/ZenFPGA/chisel/target/scala-2.12/classes ...
[warn] there were 5 feature warnings; re-run with -feature for details
[warn] one warning found
[info] Done compiling.
[info] Compiling 1 Scala source to /home/jcondley/Zendar/kodo/ZenFPGA/chisel/target/scala-2.12/test-classes ...
[warn] there were two deprecation warnings (since chisel3, will be removed by end of 2017); re-run with -deprecation for details
[warn] there were two feature warnings; re-run with -feature for details
[warn] two warnings found
[info] Done compiling.
[info] [0.004] Elaborating design...
[deprecated] DspComplex.scala:22 (1029 calls): isLit is deprecated: "isLit is deprecated, use litOption.isDefined"
[deprecated] DspComplex.scala:22 (1029 calls): litArg is deprecated: "litArg is deprecated, use litOption or litTo*Option"
[deprecated] DspComplex.scala:23 (1029 calls): isLit is deprecated: "isLit is deprecated, use litOption.isDefined"
[deprecated] DspComplex.scala:23 (1029 calls): litArg is deprecated: "litArg is deprecated, use litOption or litTo*Option"
[warn] There were 4 deprecated function(s) used. These may stop compiling in a future release - you are encouraged to fix these issues.
[warn] Line numbers for deprecations reported by Chisel may be inaccurate; enable scalac compiler deprecation warnings via either of the following methods:
[warn] In the sbt interactive console, enter:
[warn] set scalacOptions in ThisBuild ++= Seq("-unchecked", "-deprecation")
[warn] or, in your build.sbt, add the line:
[warn] scalacOptions := Seq("-unchecked", "-deprecation")
[info] [1.487] Done elaborating.
Total FIRRTL Compile Time: 1887.8 ms
Total FIRRTL Compile Time: 770.6 ms
End of dependency graph
Circuit state created
[info] TaylorTest:
[info] TaylorWindow
[info] taylor.TaylorTest *** ABORTED ***
[info] java.lang.StackOverflowError:
[info] at firrtl_interpreter.LoFirrtlExpressionEvaluator.evaluate(LoFirrtlExpressionEvaluator.scala:264)
[info] at firrtl_interpreter.LoFirrtlExpressionEvaluator.$anonfun$resolveDependency$1(LoFirrtlExpressionEvaluator.scala:453)
[info] at firrtl_interpreter.Timer.apply(Timer.scala:40)
[info] at firrtl_interpreter.LoFirrtlExpressionEvaluator.resolveDependency(LoFirrtlExpressionEvaluator.scala:445)
[info] at firrtl_interpreter.LoFirrtlExpressionEvaluator.getValue(LoFirrtlExpressionEvaluator.scala:81)
[info] at firrtl_interpreter.LoFirrtlExpressionEvaluator.evaluate(LoFirrtlExpressionEvaluator.scala:304)
[info] at firrtl_interpreter.LoFirrtlExpressionEvaluator.$anonfun$resolveDependency$1(LoFirrtlExpressionEvaluator.scala:453)
[info] at firrtl_interpreter.Timer.apply(Timer.scala:40)
[info] at firrtl_interpreter.LoFirrtlExpressionEvaluator.resolveDependency(LoFirrtlExpressionEvaluator.scala:445)
[info] at firrtl_interpreter.LoFirrtlExpressionEvaluator.getValue(LoFirrtlExpressionEvaluator.scala:81)
[info] ...
This looks suspiciously like another ROM issue from here:
https://github.com/freechipsproject/chisel3/issues/642
but trying Chick's response here:
export SBT_OPTS="-Xmx2G -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=2G -Xss2M -Duser.timezone=GMT"
seems to not solve the issue (and one of the options, MaxPermSize is ignored)
Is this a legitimate Chisel bug with ROMs or is something else going on here?
Actual module with the ROM:
package taylor
import chisel3._
import chisel3.util._
import chisel3.experimental.FixedPoint
import dsptools.numbers._
import scala.io.Source
class TaylorWindow(len: Int, window: Seq[FixedPoint]) extends Module {
val io = IO(new Bundle {
val d_valid_in = Input(Bool())
val sample = Input(DspComplex(FixedPoint(16.W, 8.BP), FixedPoint(16.W, 8.BP)))
val windowed_sample = Output(DspComplex(FixedPoint(32.W, 8.BP), FixedPoint(32.W, 8.BP)))
val d_valid_out = Output(Bool())
})
val win_coeff = VecInit(window.map(x=>DspComplex.wire(x, FixedPoint(0, 16.W, 8.BP))).toSeq) // ROM storing our coefficients.
io.d_valid_out := io.d_valid_in
val counter = RegInit(UInt(10.W), 0.U)
// Implicit reset
io.windowed_sample:= io.sample * win_coeff(counter)
when(io.d_valid_in) {
counter := counter + 1.U
}
}
object TaylorDriver extends App {
val filename = "src/test/test_data/taylor_coeffs"
val coeff_file = Source.fromFile(filename).getLines
val double_coeffs = coeff_file.map(x => x.toDouble)
val fp_coeffs = double_coeffs.map(x => FixedPoint.fromDouble(x, 16.W, 8.BP))
val fp_seq = fp_coeffs.toSeq
chisel3.Driver.execute(args, () => new TaylorWindow(1024, fp_seq))
}
Tester Code:
package taylor
import chisel3._
import chisel3.util._
import chisel3.experimental.FixedPoint
import dsptools.numbers.implicits._
import scala.io.Source
import chisel3.iotesters
import chisel3.iotesters.{ChiselFlatSpec, Driver, PeekPokeTester}
class TaylorWindowUnitTest(dut: TaylorWindow) extends PeekPokeTester(dut) {
val filename = "src/test/test_data/taylor_coeffs"
val coeff_file = Source.fromFile(filename).getLines
val double_coeffs = coeff_file.map(x => x.toDouble)
val fp_coeffs = double_coeffs.map(x => FixedPoint.fromDouble(x, 16.W, 8.BP))
val fp_seq = fp_coeffs.toSeq
poke(dut.io.d_valid_in, Bool(false))
expect(dut.io.d_valid_out, Bool(false))
}
class TaylorTest extends ChiselFlatSpec {
behavior of "TaylorWindow"
backends foreach {backend =>
it should s"test the basic Taylow Window" in {
Driver(() => new TaylorWindow(1024, getSeq()), backend)(c => new TaylorWindowUnitTest(c)) should be (true)
}
}
def getSeq() : Seq[FixedPoint] = {
val filename = "src/test/test_data/taylor_coeffs"
val coeff_file = Source.fromFile(filename).getLines
val double_coeffs = coeff_file.map(x => x.toDouble)
val fp_coeffs = double_coeffs.map(x => FixedPoint.fromDouble(x, 16.W, 8.BP))
fp_coeffs.toSeq
}
}

This looks like a failure in the firrtl-interpreter (one of the Scala based Chisel simulator) which can have problems with a number of large firrtl constructs. If you have verilator installed can you try changing
backends foreach {backend =>
to
Seq("verilator") foreach {backend =>
and see what happens. Another think to try is Treadle which is the new version of the interpreter and is not in full release yet but is available in snapshot versions of the chisel ecosystem. It should be able to handle this also.

Related

How to run Karate and Gatling with Gradle build system for performance testing

I'm trying to run a Karate test as a Gatling performance test. I am being use Gradle however. When trying to run under Gradle the below disaster unfolds.
Appreciate any ideas what might be causing KarateAction to crash.
i have added Gatling dependencies in gradle build
BUILD GRADLE:
plugins {
// id 'org.springframework.boot' version '2.5.5'
// id 'io.spring.dependency-management' version '1.0.11.RELEASE'
id 'java'
id 'net.researchgate.release' version '2.8.1'
}
group 'org.ikano'
version '1.0-SNAPSHOT'
sourceCompatibility = 1.8
ext {
cucumberVersion = "5.7.0"
karateVersion = '1.1.0'
// kafkaProducer = "3.2.0"
// kafkaConsumer = "3.2.0"
// kafkaCommonModel = "3.2.0"
dbUtilsVersion = "1.7"
}
apply plugin: 'scala'
configurations {
gatling
}
sourceSets {
test {
resources {
srcDir file('src/test/java')
exclude '**/*.java'
exclude '**/*.scala'
}
}
}
test {
useJUnitPlatform()
systemProperty "karate.options", System.properties.getProperty("karate.options")
systemProperty "karate.env", System.properties.getProperty("karate.env")
outputs.upToDateWhen { false }
}
repositories {
maven {
url "https://artifacts.le.tnt.bank.*****/repository/maven-****"
credentials {
username '*******'
password '**********'
}
mavenCentral()
}
}
task karateDebug(type: JavaExec) {
classpath = sourceSets.test.runtimeClasspath
main = 'com.intuit.karate.cli.Main'
}
dependencies {
implementation("commons-configuration:commons-configuration:1.10")
implementation("org.json:json:20180813")
implementation("javax.mail:mail:1.5.0-b01")
implementation("com.jayway.jsonpath:json-path:2.6.0")
implementation("com.googlecode.json-simple:json-simple:1.1.1")
// implementation("org.springframework:spring-context:5.3.13")
implementation("org.apache.poi:poi-ooxml:4.1.1")
// implementation("org.springframework:spring-webmvc:4.3.0.RELEASE")
testImplementation("com.intuit.karate:karate-core:1.2.0")
testImplementation("junit:junit:4.13.2")
testImplementation("com.intuit.karate:karate-junit5:1.1.0")
testImplementation("com.intuit.karate:karate-gatling:1.1.0")
testImplementation("com.intuit.karate:karate-apache:0.9.6")
testImplementation("net.masterthought:cucumber-reporting:3.8.0")
testImplementation("hu.blackbelt.bundles.poi:org.apache.poi:4.1.2_2")
testImplementation("org.apache.poi:poi:4.1.1")
testImplementation("io.rest-assured:rest-assured:4.3.0")
testCompile 'com.intuit.karate:karate-junit5:1.1.0'
testCompile "com.intuit.karate:karate-gatling:${karateVersion}"
// compile group: 'org.springframework.boot', name: 'spring-boot-starter-parent', version: '2.5.5', ext: 'pom'
//implementation group: 'org.springframework', name: 'spring-websocket', version: '5.2.9.RELEASE'
//implementation group: 'org.springframework', name: 'spring-messaging', version: '5.2.9.RELEASE'
// implementation 'org.springframework.boot:spring-boot-starter-web'
// implementation 'org.springframework.boot:spring-boot-starter'
// testImplementation('org.springframework.boot:spring-boot-starter-test')
//{exclude group: 'org.junit.vintage', module: 'junit-vintage-engine'}
compile group: 'joda-time', name: 'joda-time', version: '2.10.6'
// implementation "com.ikano.ipay.eem.kafka:kafka-producer:${kafkaProducer}"
// implementation "com.ikano.ipay.eem.kafka:kafka-consumer:${kafkaConsumer}"
// implementation "com.ikano.ipay.eem.kafka:kafka-common-model:${kafkaCommonModel}"
implementation group: 'redis.clients', name: 'jedis', version: '2.9.0'
implementation group: 'org.redisson', name: 'redisson', version: "3.16.3"
compile group: 'org.apache.poi', name: 'poi', version: '4.1.2'
compile group: 'org.apache.poi', name: 'poi-ooxml', version: '4.1.2'
compile group: 'commons-dbutils', name: 'commons-dbutils', version: "$dbUtilsVersion"
task gatlingRun(type: JavaExec) {
group = 'Web Tests'
description = 'Run Gatling Tests'
new File("${buildDir}/reports/gatling").mkdirs()
classpath = sourceSets.test.runtimeClasspath
main = "io.gatling.app.Gatling"
args = [
// change this to suit your simulation entry-point
'-s', 'performance2.userSimulation',
'-rf', "${buildDir}/reports/gatling"
]
systemProperties System.properties
}
group = "org.ikano"
version = "1.0-SNAPSHOT"
description = "IPay"
java.sourceCompatibility = JavaVersion.VERSION_1_8
FOR SCALA
package performance
import com.intuit.karate.gatling.PreDef._
import io.gatling.core.Predef._
import io.gatling.core.structure.ScenarioBuilder
import scala.concurrent.duration._
class UserSimulation extends Simulation{
val getSingleUser: ScenarioBuilder = scenario("UK Onboarding - Verify user is able to onboard successfully").exec(karateFeature("classpath:performance/onboarding.feature"))
setUp(
getSingleUser.inject(rampUsers(10).during(5 seconds))
)
}
Command Line :gradle gatlingRun
Starting a Gradle Daemon (subsequent builds will be faster)
> Task :gatlingRun
16:10:18.039 [main] INFO i.g.c.config.GatlingConfiguration$ - Gatling will try to use 'gatling.conf' as config file.
16:10:20.052 [GatlingSystem-akka.actor.default-dispatcher-6] INFO akka.event.slf4j.Slf4jLogger - Slf4jLogger started
16:10:22.818 [main] ERROR io.gatling.app.Gatling$ - Run crashed
java.lang.IllegalArgumentException: User defined Simulation class performance2.userSimulation could not be loaded
at io.gatling.app.Selection$Selector.findUserDefinedSimulationInClassloader$1(Selection.scala:80)
at io.gatling.app.Selection$Selector.$anonfun$singleSimulationFromConfig$4(Selection.scala:85)
at scala.Option.orElse(Option.scala:477)
at io.gatling.app.Selection$Selector.$anonfun$singleSimulationFromConfig$3(Selection.scala:85)
at scala.Option.flatMap(Option.scala:283)
at io.gatling.app.Selection$Selector.singleSimulationFromConfig(Selection.scala:83)
at io.gatling.app.Selection$Selector.$anonfun$selection$1(Selection.scala:52)
at scala.Option.getOrElse(Option.scala:201)
at io.gatling.app.Selection$Selector.selection(Selection.scala:44)
at io.gatling.app.Selection$.apply(Selection.scala:36)
at io.gatling.app.Runner.run0(Runner.scala:61)
at io.gatling.app.Runner.run(Runner.scala:49)
at io.gatling.app.Gatling$.start(Gatling.scala:83)
at io.gatling.app.Gatling$.fromArgs(Gatling.scala:45)
at io.gatling.app.Gatling$.main(Gatling.scala:37)
at io.gatling.app.Gatling.main(Gatling.scala)
Caused by: java.lang.ClassNotFoundException: performance2.userSimulation
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at io.gatling.app.Selection$Selector.$anonfun$singleSimulationFromConfig$2(Selection.scala:71)
at scala.util.Try$.apply(Try.scala:210)
at io.gatling.app.Selection$Selector.findUserDefinedSimulationInClassloader$1(Selection.scala:71)
... 15 common frames omitted
Exception in thread "main" java.lang.IllegalArgumentException: User defined Simulation class performance2.userSimulation could not be loaded
at io.gatling.app.Selection$Selector.findUserDefinedSimulationInClassloader$1(Selection.scala:80)
at io.gatling.app.Selection$Selector.$anonfun$singleSimulationFromConfig$4(Selection.scala:85)
at scala.Option.orElse(Option.scala:477)
at io.gatling.app.Selection$Selector.$anonfun$singleSimulationFromConfig$3(Selection.scala:85)
at scala.Option.flatMap(Option.scala:283)
at io.gatling.app.Selection$Selector.singleSimulationFromConfig(Selection.scala:83)
at io.gatling.app.Selection$Selector.$anonfun$selection$1(Selection.scala:52)
at scala.Option.getOrElse(Option.scala:201)
at io.gatling.app.Selection$Selector.selection(Selection.scala:44)
at io.gatling.app.Selection$.apply(Selection.scala:36)
at io.gatling.app.Runner.run0(Runner.scala:61)
at io.gatling.app.Runner.run(Runner.scala:49)
at io.gatling.app.Gatling$.start(Gatling.scala:83)
at io.gatling.app.Gatling$.fromArgs(Gatling.scala:45)
at io.gatling.app.Gatling$.main(Gatling.scala:37)
at io.gatling.app.Gatling.main(Gatling.scala)
Caused by: java.lang.ClassNotFoundException: performance2.userSimulation
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at io.gatling.app.Selection$Selector.$anonfun$singleSimulationFromConfig$2(Selection.scala:71)
at scala.util.Try$.apply(Try.scala:210)
at io.gatling.app.Selection$Selector.findUserDefinedSimulationInClassloader$1(Selection.scala:71)
... 15 more
> Task :gatlingRun FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':gatlingRun'.
> Process 'command 'C:\Program Files\Amazon Corretto\jdk1.8.0_282\bin\java.exe'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 44s
3 actionable tasks: 1 executed, 2 up-to-date
The package name is performance but inside the log it says:
performance2.userSimulation
Can you change it to in the run command:
performance.userSimulation
I think that is the issue you are facing. It cant find the simulation class.

Use SparkSession.sql() with JDBC

Problem :
I would like to use JDBC connection to make a custom request using spark.
The goal of this query is to optimized memory allocation on workers, because of that I can't use :
ss.read
.format("jdbc")
.option("url", "jdbc:postgresql:dbserver")
.option("dbtable", "schema.tablename")
.option("user", "username")
.option("password", "password")
.load()
Currently :
I am currently trying to run :
ss = SparkSession
.builder()
.appName(appName)
.master("local")
.config(conf)
.getOrCreate()
ss.sql("some custom query")
Configuration :
url=jdbc:mysql://127.0.0.1/database_name
driver=com.mysql.jdbc.Driver
user=user_name
password=xxxxxxxxxx
Error :
[info] Exception encountered when attempting to run a suite with class name: db.TestUserProvider *** ABORTED ***
[info] org.apache.spark.sql.AnalysisException: Table or view not found: users; line 1 pos 14
[info] at org.apache.spark.sql.catalyst.analysis.package$AnalysisErrorAt.failAnalysis(package.scala:42)
[info] at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$.org$apache$spark$sql$catalyst$analysis$Analyzer$ResolveRelations$$lookupTableFromCatalog(Analyzer.scala:459)
[info] at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$$anonfun$apply$8.applyOrElse(Analyzer.scala:478)
[info] at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$$anonfun$apply$8.applyOrElse(Analyzer.scala:463)
[info] at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan$$anonfun$resolveOperators$1.apply(LogicalPlan.scala:61)
[info] at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan$$anonfun$resolveOperators$1.apply(LogicalPlan.scala:61)
[info] at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:70)
[info] at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.resolveOperators(LogicalPlan.scala:60)
[info] at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan$$anonfun$1.apply(LogicalPlan.scala:58)
[info] at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan$$anonfun$1.apply(LogicalPlan.scala:58)
Assumption :
I guess there is a configuration error, but I can't find out where.
Spark can read and write data to/from relational databases using the JDBC data source (like you did in your first code example).
In addition (and completely separately), spark allows using SQL to query views that were created over data that was already loaded into a DataFrame from some source. For example:
val df = Seq(1,2,3).toDF("a") // could be any DF, loaded from file/JDBC/memory...
df.createOrReplaceTempView("my_spark_table")
spark.sql("select a from my_spark_table").show()
Only "tables" (called views, as of Spark 2.0.0) created this way can be queried using SparkSession.sql.
If your data is stored in a relational database, Spark will have to read it from there first, and only then would it be able to execute any distributed computation on the loaded copy. Bottom line - we can load the data from the table using read, create a temp view, and then query it:
ss.read
.format("jdbc")
.option("url", "jdbc:mysql://127.0.0.1/database_name")
.option("dbtable", "schema.tablename")
.option("user", "username")
.option("password", "password")
.load()
.createOrReplaceTempView("my_spark_table")
// and then you can query the view:
val df = ss.sql("select * from my_spark_table where ... ")

Chisel AlreadyBoundException

I am developing a simple on chip memory for an SoC, based on Sodor scratchpad memory. So, first I'm converting a slightly modified version of that design to chisel 3. Now, I'm getting this exception regarding a bounded type that I can't understand.
[info] - should correctly write and read data *** FAILED ***
[info] chisel3.core.Binding$BindingException: Error: Cannot set as output .M_WR: Already bound to LitBinding()
[info] at chisel3.core.Binding$.bind(Binding.scala:100)
[info] at chisel3.core.Output$.apply(Data.scala:50)
[info] at chisel3.util.ReadyValidIO.<init>(Decoupled.scala:22)
[info] at chisel3.util.DecoupledIO.<init>(Decoupled.scala:72)
[info] at chisel3.util.Decoupled$.apply(Decoupled.scala:81)
[info] at mem.MemPortIO.<init>(memory.scala:40)
[info] at mem.OnChipMemory$$anon$1.<init>(memory.scala:49)
[info] at mem.OnChipMemory.<init>(memory.scala:47)
[info] at mem.memoryTester$$anonfun$3$$anonfun$apply$1$$anonfun$apply$mcV$sp$1.apply(memoryTest.scala:33)
[info] at mem.memoryTester$$anonfun$3$$anonfun$apply$1$$anonfun$apply$mcV$sp$1.apply(memoryTest.scala:33)
[info] at chisel3.core.Module$.do_apply(Module.scala:35)
[info] at chisel3.Driver$$anonfun$elaborate$1.apply(Driver.scala:194)
[info] at chisel3.Driver$$anonfun$elaborate$1.apply(Driver.scala:194)
[info] at chisel3.internal.Builder$$anonfun$build$1.apply(Builder.scala:206)
[info] at chisel3.internal.Builder$$anonfun$build$1.apply(Builder.scala:204)
[info] at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
[info] at chisel3.internal.Builder$.build(Builder.scala:204)
[info] at chisel3.Driver$.elaborate(Driver.scala:194)
[info] at chisel3.Driver$.execute(Driver.scala:229)
[info] at chisel3.iotesters.setupFirrtlTerpBackend$.apply(FirrtlTerpBackend.scala:110)
[info] at chisel3.iotesters.Driver$.execute(Driver.scala:47)
[info] at chisel3.iotesters.Driver$.apply(Driver.scala:210)
[info] at mem.memoryTester$$anonfun$3$$anonfun$apply$1.apply$mcV$sp(memoryTest.scala:33)
[info] at mem.memoryTester$$anonfun$3$$anonfun$apply$1.apply(memoryTest.scala:33)
[info] at mem.memoryTester$$anonfun$3$$anonfun$apply$1.apply(memoryTest.scala:33)
[info] at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
[info] at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
[info] at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
[info] at org.scalatest.Transformer.apply(Transformer.scala:22)
[info] at org.scalatest.Transformer.apply(Transformer.scala:20)
[info] at org.scalatest.FlatSpecLike$$anon$1.apply(FlatSpecLike.scala:1647)
[info] at org.scalatest.Suite$class.withFixture(Suite.scala:1122)
[info] at org.scalatest.FlatSpec.withFixture(FlatSpec.scala:1683)
[info] at org.scalatest.FlatSpecLike$class.invokeWithFixture$1(FlatSpecLike.scala:1644)
[info] at org.scalatest.FlatSpecLike$$anonfun$runTest$1.apply(FlatSpecLike.scala:1656)
[info] at org.scalatest.FlatSpecLike$$anonfun$runTest$1.apply(FlatSpecLike.scala:1656)
[info] at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
[info] at org.scalatest.FlatSpecLike$class.runTest(FlatSpecLike.scala:1656)
[info] at org.scalatest.FlatSpec.runTest(FlatSpec.scala:1683)
[info] at org.scalatest.FlatSpecLike$$anonfun$runTests$1.apply(FlatSpecLike.scala:1714)
[info] at org.scalatest.FlatSpecLike$$anonfun$runTests$1.apply(FlatSpecLike.scala:1714)
[info] at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:413)
[info] at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:401)
[info] at scala.collection.immutable.List.foreach(List.scala:381)
[info] at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
[info] at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:390)
[info] at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:427)
[info] at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:401)
[info] at scala.collection.immutable.List.foreach(List.scala:381)
[info] at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
[info] at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:396)
[info] at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:483)
[info] at org.scalatest.FlatSpecLike$class.runTests(FlatSpecLike.scala:1714)
[info] at org.scalatest.FlatSpec.runTests(FlatSpec.scala:1683)
[info] at org.scalatest.Suite$class.run(Suite.scala:1424)
[info] at org.scalatest.FlatSpec.org$scalatest$FlatSpecLike$$super$run(FlatSpec.scala:1683)
[info] at org.scalatest.FlatSpecLike$$anonfun$run$1.apply(FlatSpecLike.scala:1760)
[info] at org.scalatest.FlatSpecLike$$anonfun$run$1.apply(FlatSpecLike.scala:1760)
[info] at org.scalatest.SuperEngine.runImpl(Engine.scala:545)
[info] at org.scalatest.FlatSpecLike$class.run(FlatSpecLike.scala:1760)
[info] at org.scalatest.FlatSpec.run(FlatSpec.scala:1683)
[info] at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:466)
[info] at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:677)
[info] at sbt.TestRunner.runTest$1(TestFramework.scala:76)
[info] at sbt.TestRunner.run(TestFramework.scala:85)
[info] at sbt.TestFramework$$anon$2$$anonfun$$init$$1$$anonfun$apply$8.apply(TestFramework.scala:202)
[info] at sbt.TestFramework$$anon$2$$anonfun$$init$$1$$anonfun$apply$8.apply(TestFramework.scala:202)
[info] at sbt.TestFramework$.sbt$TestFramework$$withContextLoader(TestFramework.scala:185)
[info] at sbt.TestFramework$$anon$2$$anonfun$$init$$1.apply(TestFramework.scala:202)
[info] at sbt.TestFramework$$anon$2$$anonfun$$init$$1.apply(TestFramework.scala:202)
[info] at sbt.TestFunction.apply(TestFramework.scala:207)
[info] at sbt.Tests$$anonfun$9.apply(Tests.scala:216)
[info] at sbt.Tests$$anonfun$9.apply(Tests.scala:216)
[info] at sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:44)
[info] at sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:44)
[info] at sbt.std.Transform$$anon$4.work(System.scala:63)
[info] at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
[info] at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
[info] at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
[info] at sbt.Execute.work(Execute.scala:237)
[info] at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
[info] at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
[info] at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
[info] at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
[info] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[info] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[info] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[info] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[info] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[info] at java.lang.Thread.run(Thread.java:745)
[info] ScalaCheck
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] ScalaTest
I'm not clear about the value that is causing the problem. First I thought it was a problem with the Vec's I use in the Mem and IO ports (because of an explanation I found for a similar exception). But the problem exists even after I get rid of them. I even tried getting rid of clone functions.
Now, I'm clueless about the cause of the problem. I'm posting the main section of the source code (except parameter declarations and few simple functions) for the memory in case it might be useful.
trait MemOpConstants
{
val MT_X = Bits(0, 3) // memory transfer type
val MT_B = Bits(1, 3)
val MT_H = Bits(2, 3)
val MT_W = Bits(3, 3)
val MT_D = Bits(4, 3)
val MT_BU = Bits(5, 3)
val MT_HU = Bits(6, 3)
val MT_WU = Bits(7, 3)
val M_X = Bits("b0", 1) // access type
val M_RD = Bits("b0", 1) // load
val M_WR = Bits("b1", 1) // store
}
class MemReq(data_width: Int)(implicit config: Configuration) extends Bundle with MemOpConstants
{
val addr = UInt(width = config.xprlen)
val data = Bits(width = data_width)
val fcn = Bits(width = M_X.getWidth) // memory function code
val typ = Bits(width = MT_X.getWidth) // memory access type
override def cloneType = { new MemReq(data_width).asInstanceOf[this.type] }
}
class MemResp(data_width: Int) extends Bundle
{
val data = Bits(width = data_width)
override def cloneType = { new MemResp(data_width).asInstanceOf[this.type] }
}
class MemPortIO(data_width: Int)(implicit config: Configuration) extends Bundle // constructor for IO interface of data memory
{
val req = Decoupled(new MemReq(data_width)) // ready valid pair
val resp = (new ValidIO(new MemResp(data_width))).flip // valid signal
override def cloneType = { new MemPortIO(data_width).asInstanceOf[this.type] }
}
class OnChipMemory(num_ports: Int = 2, num_bytes: Int = (1 << 15), seq_read: Boolean = false)(implicit config: Configuration) extends Module with MemOpConstants
{
val io = IO(new Bundle{
val port = Vec(num_ports, (new MemPortIO(data_width = config.xprlen)).flip)
})
val num_bytes_per_line = 4
val num_lines = num_bytes/num_bytes_per_line
val lsb_idx = log2Up(num_bytes_per_line) // index of lsb in address
val chipMem = Mem(Vec(4, UInt(width = 32)), num_lines) // memory
for (i <- 0 until num_ports)
{
io.port(i).resp.valid := Reg(next = io.port(i).req.valid)
io.port(i).req.ready := Bool(true) // for now
val req_valid = io.port(i).req.valid
val req_addr = io.port(i).req.bits.addr
val req_data = io.port(i).req.bits.data
val req_fn = io.port(i).req.bits.fcn
val req_typ = io.port(i).req.bits.typ
val byte_shift_amt = io.port(i).req.bits.addr(1,0)
val bit_shift_amt = Cat(byte_shift_amt, UInt(0,3))
//mem read
val r_data_idx = Reg(UInt())
val data_idx = req_addr >> UInt(lsb_idx)
val read_data = Bits()
val rdata = Bits()
if (seq_read)
{
read_data := chipMem(r_data_idx)
rdata := LoadDataGen((read_data >> Reg(next=bit_shift_amt)), Reg(next=req_typ))
}
else
{
read_data := chipMem(data_idx)
rdata := LoadDataGen((read_data >> bit_shift_amt), req_typ)
}
io.port(i).resp.bits.data := rdata
//mem write
when (req_valid && req_fn === M_WR)
{
val wdata = StoreDataGen(req_data, req_typ)
val wmask = ((StoreMask(req_typ) << bit_shift_amt)(31,0)).toBools
chipMem.write(data_idx, wdata, wmask)
}
.elsewhen (req_valid && req_fn === M_RD)
{
r_data_idx := data_idx
}
}
}
object StoreDataGen extends MemOpConstants
{
def apply(din: Bits, typ: Bits): Vec[UInt] =
{
val word = (typ.equals(MT_W)) || (typ.equals(MT_WU))
val half = (typ.equals(MT_H)) || (typ.equals(MT_HU))
val byte = (typ.equals(MT_B)) || (typ.equals(MT_BU))
val dout = Mux(Bool(byte), Vec(4, din( 7,0)),
Mux(Bool(half), Vec(din(15,8), din( 7,0), din(15,8), din( 7,0)),
Vec(din(31,25), din(24,16), din(15,8), din( 7,0))))
return dout
}
}
It would help to have more of the code with line numbers, but based on the name of the trait MemOpConstants, I would guess that it includes some Chisel literals. The compiler is complaining it cannot make an IO port out of MemPortIO since it contains a Bundle MemReq which contains literals as elements.
IO Ports may only contain pure, unbound Chisel data types, not literals, Reg's, Mem's, or Wire's.
If you want to make the constants available at the IO Ports, you should define slots (UInts) to receive them, and then wire those slots up to the actual constant/literal value. Don't place the constants themselves in the IO Port definition.
All your definitions in MemOpConstants are literals (constants) of type Bits. They have values and widths. The values are what make them literals (constants), and unsuitable for inclusion in an IO Port definition.

Best way to pass the schema name as a variable to a query

I have a PlayFramework server (with Anorm) which operates against a database with several schemas, all of them with the same tables.
Most of my "access to database" functions look like:
def findById(zoneName: String, id: Long): Option[Employee] = {
DB.withConnection { implicit connection =>
SQL("""select *
from """+zoneName+"""employee
where employee._id = {id}"""
.on(
'_id -> id
).as(simpleParser.singleOpt)
}
}
But I know this is a wrong approach, because it is not SQL-Injection-safe and of course it is tedious to write in every function.
I want to use String interpolation to correct this, it works well with my id variable, but it doesn't with zoneName:
def findById(zoneName: String, id: Long): Option[Employee] = {
DB.withConnection { implicit connection =>
SQL"""select *
from $zoneName.employee
where employee._id = 1"""
.as(simpleParser.singleOpt)
}
}
Gives me:
info] ! #6lenhal6c - Internal server error, for (GET) [/limbo/br/employee/1] ->
[info]
[info] play.api.Application$$anon$1: Execution exception[[PSQLException: ERROR: syntax error at or near «$1»
[info] Position: 25]]
[info] at play.api.Application$class.handleError(Application.scala:296) ~[play_2.11-2.3.8.jar:2.3.8]
[info] at play.api.DefaultApplication.handleError(Application.scala:402) [play_2.11-2.3.8.jar:2.3.8]
[info] at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$3$$anonfun$applyOrElse$4.apply(PlayDefaultUpstreamHandler.scala:320) [play_2.11-2.3.8.jar:2.3.8]
[info] at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$3$$anonfun$applyOrElse$4.apply(PlayDefaultUpstreamHandler.scala:320) [play_2.11-2.3.8.jar:2.3.8]
[info] at scala.Option.map(Option.scala:146) [scala-library-2.11.5.jar:na]
[info] Caused by: org.postgresql.util.PSQLException: ERROR: syntax error at or near «$1»
[info] Position: 25
[info] at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2198) ~[postgresql-9.3-1102.jdbc4.jar:na]
[info] at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1927) ~[postgresql-9.3-1102.jdbc4.jar:na]
[info] at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) ~[postgresql-9.3-1102.jdbc4.jar:na]
[info] at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:561) ~[postgresql-9.3-1102.jdbc4.jar:na
Tested also with ${zoneName} with the same result.
Any help or advice about how to write this would be appreciated, thank you in advance!
Using Anorm String interpolation, any $expression is to be provided a parameter, that is to say if it's a string it will quoted by the JDBC driver.
If you want to substitute part of the SQL statement with string (e.g. dynamic schema), either you can use concatenation, or since latest versions (2.4.0-M3 or 2.3.8) the syntax #$expr.
val table = "myTable"
SQL"SELECT * FROM #$table WHERE id=$id"
// SELECT * FROM myTable WHERE id=?

Testing json data using Specs2

I added(In built.sbt) matcher-extra :-
"org.specs2" %% "specs2" % "2.3.4" % "test",
"org.specs2" % "specs2-matcher-extra_2.10" % "2.3-scalaz-7.1.0-M3",
the ("/" the symbols are not resolving)
My example test case for Json is looking like below:-
package specs.model
import org.specs2.mutable.Specification
import org.specs2.matcher.JsonMatchers
class Json extends Specification with JsonMatchers {
"Json Matcher" should {
"1st field" in {
val json = """{"name":"sagar"}"""
json must /("name" -> "sagar")
}
"2nd field" in {
val json = """{"id":1}"""
json must /("id" -> 1.0)
}
}
}
ErrorMsg:-
[info] Compiling 2 Scala sources to \target\scala- 2.10\test-classes...
[info] Json
[info]
[info] Json Matcher should
[info] + 1st field
[info] + 2nd field
[info]
[info] Total for specification Json
[info] Finished in 76 ms
[info] 2 examples, 0 failure, 0 error
[trace] Stack trace suppressed: run 'last specBuilder/test:test' for the full output.
[error] Could not run test specs.model.Json: java.lang.NoSuchMethodError: scalaz.Scalaz$.tuple2Monoid(Lscalaz/Monoid;Lscalaz/Monoid;)Lscalaz/std/Tuple2Monoid;
[error] Error: Total 0, Failed 0, Errors 0, Passed 0, Skipped 0
[error] Error during tests:
[error] specs.model.Json
[error] (specBuilder/test:test) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 9 s, completed 11 Dec, 2013 5:12:39 PM
I am strucking here please give me the solution
Thanks,
GSY
I finally got it to work.
There are some instructions for setting up your build.sbt file here. Scroll down to the very bottom of the page. http://etorreborre.github.io/specs2/#Downloads
The import commands in my Application.spec file are:
import org.specs2.mutable._
import org.specs2.mutable.Specification
import org.specs2.matcher.JsonMatchers
import org.specs2.runner._
class yourClass extends Specification with JsonMatchers { }
The following jar files are installed in the lib directory
specs2_2.10-2.3.7-javadoc.jar
scalaz-core_2.10-7.0.4-javadoc.jar
scalaz-concurrent_2.10-7.0.4-javadoc.jar
This is the contents of my build.sbt file.
name := "playExperiments"
version := "1.0-SNAPSHOT"
libraryDependencies ++= Seq(
"org.specs2" %% "specs2" % "2.3.7" % "test",
jdbc,
anorm,
cache
)
scalacOptions in Test ++= Seq("-Yrangepos")
resolvers ++= Seq("snapshots", "releases").map(Resolver.sonatypeRepo)
play.Project.playScalaSettings