I've been using Julia for some graphical results, using ODBC to conect MS Access database to get the data.
The same function worked flawlesly two weeks ago, but now it throws an error:
ERROR: MethodError: no method matching eachcolumn(::Tables.CopiedColumns{NamedTuple{(:year, :Fact),Tuple{Array{Union{Missing, Int16},1},Array{Union{Missing, Float64},1}}}})
Closest candidates are:
eachcolumn(::Union{Function, Type}, ::Tables.Schema{names,nothing}, ::Any) where names at C:\Users\myuser\.julia\packages\Tables\TA7NF\src\utils.jl:109
eachcolumn(::Union{Function, Type}, ::Tables.Schema{names,types}, ::Any) where {names, types} at C:\Users\myuser\.julia\packages\Tables\TA7NF\src\utils.jl:66
Stacktrace:
[1] #fromcolumns#410(::Bool, ::typeof(DataFrames.fromcolumns), ::Tables.CopiedColumns{NamedTuple{(:anno, :Fact),Tuple{Array{Union{Missing, Int16},1},Array{Union{Missing, Float64},1}}}}) at C:\Users\myuser\.julia\packages\DataFrames\yH0f6\src\other\tables.jl:13
[2] (::DataFrames.var"#kw##fromcolumns")(::NamedTuple{(:copycols,),Tuple{Bool}}, ::typeof(DataFrames.fromcolumns), ::Tables.CopiedColumns{NamedTuple{(:anno, :Fact),Tuple{Array{Union{Missing, Int16},1},Array{Union{Missing,
Float64},1}}}}) at .\none:0
[3] #DataFrame#412(::Bool, ::Type{DataFrame}, ::ODBC.Query{missing,NamedTuple{(:anno, :Fact),Tuple{Union{Missing, Int16},Union{Missing, Float64}}},Tuple{Array{Union{Missing, Int16},1},Array{Union{Missing, Float64},1}}}) at C:\Users\myuser\.julia\packages\DataFrames\yH0f6\src\other\tables.jl:32
[4] DataFrame(::ODBC.Query{missing,NamedTuple{(:anno, :Fact),Tuple{Union{Missing, Int16},Union{Missing, Float64}}},Tuple{Array{Union{Missing, Int16},1},Array{Union{Missing, Float64},1}}}) at C:\Users\myuser\.julia\packages\DataFrames\yH0f6\src\other\tables.jl:23
[5] #query#15(::Bool, ::Bool, ::Dict{Int64,Function}, ::typeof(ODBC.query), ::ODBC.DSN, ::String, ::Type{DataFrame}) at C:\Users\myuser\.julia\packages\ODBC\YEzHX\src\Query.jl:390
[6] query(::ODBC.DSN, ::String, ::Type{DataFrame}) at C:\Users\myuser\.julia\packages\ODBC\YEzHX\src\Query.jl:385
[7] query(::ODBC.DSN, ::String) at C:\Users\myuser\.julia\packages\ODBC\YEzHX\src\Query.jl:376
[8] top-level scope at C:\Users\myuser\Documents\Fact.jl:94
Seems like there is some kind of incompatibility between Query.jl and tables.jl
Here is the code used:
using DataFrames
using DataStreams
using ODBC
using StatsBase
using Plots
myDNS = ODBC.DSN("Driver={Microsoft Access Driver (*.mdb, *.accdb)}; DBQ=C:/Users/myuser/Documents/Data.accdb")
strFactQuery = "SELECT YEAR(FFact) AS anno, SUM(Invoiced) AS Fact FROM Invoices GROUP BY YEAR(FFact)"
FactResults = ODBC.query(myDNS, strFactQuery)
Is there anyone with the same problem? maybe it's a Query.jl bug?
Regards
Downgrading to Tables v 0.2.11 solved the problem, but beeing able to get that version was a PITA as downgrading via pkg mannager was not possible because some dependency problems, so, all in all, I went to github, selected the version, copied the code and pasted over the library code in my environment...maybe not the ellegant way, but worked.
Related
I am trying to run a julia script on 4 csv files. But I keep getting this error:
samde#DESKTOP-V6PBDGC MINGW64 ~/mimix (master)
$ C:/Users/samde/AppData/Local/Programs/Julia/Julia-1.4.2/bin/julia.exe scripts/fit-mcmc.jl --hyper nutnet-analysis/configs/hyper.yml --monitor nutnet-analysis/configs/monitor-mimix.yml --inits nutnet-analysis/configs/inits.yml --factors 20 nutnet-analysis/test-data samnutnet-results
Reading X.csv
Reading Y.csv
Reading Z.csv
Beginning MCMC sampling
ERROR: LoadError: BoundsError: attempt to access 429×20 ArrayLogical{2} at index [430, Base.Slice(Base.OneTo(20))]
Stacktrace:
[1] throw_boundserror(::ArrayLogical{2}, ::Tuple{Int64,Base.Slice{Base.OneTo{Int64}}}) at .\abstractarray.jl:537
[2] checkbounds at .\abstractarray.jl:502 [inlined]
[3] _getindex at .\multidimensional.jl:726 [inlined]
[4] getindex at .\abstractarray.jl:980 [inlined]
[5] (::MicrobiomeMixedModels.var"#14#63")(::ArrayLogical{2}, ::Int64) at C:\Users\samde\mimix\MicrobiomeMixedModels.jl\src\models\mimix.jl:109
[6] (::var"#36#37")(::Model) at .\array.jl:0
[7] setinits!(::ArrayStochastic{2}, ::Model, ::Array{Float64,2}) at C:\Users\samde\.julia\packages\Mamba\PkMTm\src\model\dependent.jl:173
[8] setinits!(::Model, ::Dict{Symbol,Any}) at C:\Users\samde\.julia\packages\Mamba\PkMTm\src\model\initialization.jl:11
[9] setinits!(::Model, ::Array{Dict{Symbol,Any},1}) at C:\Users\samde\.julia\packages\Mamba\PkMTm\src\model\initialization.jl:24
[10] mcmc(::Model, ::Dict{Symbol,Any}, ::Array{Dict{Symbol,Any},1}, ::Int64; burnin::Int64, thin::Int64, chains::Int64, verbose::Bool) at C:\Users\samde\.julia\packages\Mamba\PkMTm\src\model\mcmc.jl:30
[11] top-level scope at C:\Users\samde\mimix\scripts\fit-mcmc.jl:150
[12] include(::Module, ::String) at .\Base.jl:377
[13] exec_options(::Base.JLOptions) at .\client.jl:288
[14] _start() at .\client.jl:484
in expression starting at C:\Users\samde\mimix\scripts\fit-mcmc.jl:105
I have tried deleting the last lines in every csv file. I tried saving as normal CSV files and CSV UTF-8 files. Running wc -l shows them all to be at the same lengths. I have looked at similar questions but have had trouble understanding the solutions. Any idea what could fix this error?
Here are the files: https://github.com/samd1993/mimixtest.git
The code works on the sample files found here: https://github.com/nsgrantham/mimix/tree/master/nutnet-analysis/reduced-data
#mbauman here is the code for mimix.jl at line 109:
108 F = Stochastic(2,
109 (F_mean, N) -> MultivariateDistribution[
110 MvNormal(F_mean[i, :], 1.0) for i in 1:N
111 ],
false
Thank you,
Sam
I found the solution to my own problem with the help of #mbauman's hint. The error was occurring due to my csv files not having the same dimensions. Thus, if anyone runs into this issue, make sure your data files are intact and there aren't any extra columns or rows. Rudimentary...but important.
Cheers,
Sam
I am just trying to restart with Julia (made some tries a couple of years ago but the libraries were still missing too much stuff).
I am now trying something really simple and can't figure out why doesn't work.
If I run these very same commands directly outside a function, I get what I want, but if I put them inside a function, I get an error when calling the read command inside my read_datafile function:
using ArgParse, ZipFile, CSV, DataFrames
function read_datafile(fp)
z = ZipFile.Reader(fp)
a = z.files[1]
df = DataFrame(CSV.File(read(a)))
return df
end
read_datafile("./folder1/test.zip")
SystemError: seek: Bad file descriptor
Stacktrace: [1] #systemerror#48 at ./error.jl:167 [inlined] [2]
systemerror at ./error.jl:167 [inlined] [3] seek at ./iostream.jl:129
[inlined] [4] read(::ZipFile.ReadableFile, ::Int64) at
/home/morgado/.julia/packages/ZipFile/fdYkP/src/ZipFile.jl:508 [5]
read at /home/morgado/.julia/packages/ZipFile/fdYkP/src/ZipFile.jl:504
[inlined] [6] read_datafile(::String) at ./In[14]:4 [7] top-level
scope at In[15]:1
EDIT:
Added more info.
using Pkg; Pkg.status()
Status `~/.julia/environments/v1.5/Project.toml`
[c7e460c6] ArgParse v1.1.1
[336ed68f] CSV v0.8.3
[a93c6f00] DataFrames v0.21.8
[92fee26a] GZip v0.5.1
[7073ff75] IJulia v1.23.1
[6f49c342] RCall v0.13.10
[fd094767] Suppressor v0.2.0
[70df011a] TableReader v0.4.0
[a5390f91] ZipFile v0.9.3
I found the answer, it's a 5 year old unsolved bug in the ZipFile package :( : https://github.com/fhs/ZipFile.jl/issues/14
Need to write the function with a global variable:
function read_datafile(fp)
global z = ZipFile.Reader(fp)
a = z.files[1]
df = DataFrame(CSV.File(read(a)))
return df
end
I am having an issue running a basic query on a sample dataset(link below)
http://kbcdn.tableausoftware.com/data/Superstore.xls
using R.
I have attached my code below.
#read file with XLConnect
path <- file.path("/Users/petergensler/Desktop/Sample - Superstore Sales.xls")
superstore <- readWorksheetFromFile(path, sheet= "Orders")
#Query
test <- sqldf("SELECT * FROM superstore WHERE 'Product Sub-Category' = 'Appliances'",)
test
The query executes fine, but it returns the following results:
[1] Row.ID Order.ID Order.Date Order.Priority Order.Quantity Sales
[7] Discount Ship.Mode Profit Unit.Price Shipping.Cost Customer.Name
[13] Province Region Customer.Segment Product.Category Product.Sub.Category Product.Name
[19] Product.Container Product.Base.Margin Ship.Date
<0 rows> (or 0-length row.names)
Is there something wrong with my attached packages that would be causing the query to run wrong, or is it something with my data? the column I am querying on seems to be fine, as it is a type character, and specifying a literal string should match the values(unless their is trailing whitespace), correct?
I am running R on Mac OS X 10.11.5 with the following session info:
session_info()
Session info -------------------------------------------------------------------------------------------
setting value
version R version 3.3.0 (2016-05-03)
system x86_64, darwin13.4.0
ui RStudio (0.99.896)
language (EN)
collate en_US.UTF-8
tz America/Chicago
date 2016-06-08
I have also attached my packages attached to the current session as well.
https://drive.google.com/open?id=0Bxhxg_yftHNubEc4NUZTUVoxa0E
Thanks for your help!
Following up on what G. Grothndieck said (use brackets for column names) I ran this and it worked for me:
#Query
test <- sqldf(x = "SELECT * FROM superstore WHERE [Product.Sub.Category] = 'Appliances'")
test
Most methods of reading in Data frames change spaces and hyphens in column names into . so you need to update that part of it.
I am trying to fetch symbols from a MySQL db using getSymbols, however the following code
library(blotter)
library(DBI)
library(RMySQL)
setDefaults(getSymbols.MySQL,user="****", password="****", dbname="quantmoddb")
currency("USD")
stock("myspy",currency="USD",multiplier=1)
getSymbols("myspy",src="MySQL")
throws
Error in (function (classes, fdef, mtable) :
unable to find an inherited method for function ‘dbConnect’ for signature ‘"character"’
I am pretty sure the db is set up correctly as the following code runs fine and returns the dataset as expected.
con <- dbConnect(RMySQL::MySQL(),user="****",password="****", dbname="quantmoddb",
host="localhost", port=3306)
db.Symbols <- DBI::dbListTables(con)
query <- paste("SELECT * from myspy ORDER BY date")
rs <- DBI::dbSendQuery(con, query)
fr <- DBI::fetch(rs, n=-1)
Any help is greatly appreciated!
The "character" method for dbConnect was removed in the 0.10 release of RMySQL. I'm not sure whether or not this was intentional. It was not mentioned in the release notes.
I am using the R package rjson to download weather data from Wunderground.com. Often I leave the program to run and there are no problems, with the data being collected fine. However, often the program stops running and I get the following error message:
Error in fromJSON(paste(raw.data, collapse = "")) : unclosed string
In addition: Warning message:
In readLines(conn, n = -1L, ok = TRUE) :
incomplete final line found on 'http://api.wunderground.com/api/[my_API_code]/history_20121214pws:1/q/pws:IBIRMING7.json'
Does anyone know what this means, and how I can avoid it since it stops my program from collecting data as I would like?
Many thanks,
Ben
I can recreate your error message using the rjson package.
Here's an example that works.
rjson::fromJSON('{"x":"a string"}')
# $x
# [1] "a string"
If we omit a double quote from the value of x, then we get the error message.
rjson::fromJSON('{"x":"a string}')
# Error in rjson::fromJSON("{\"x\":\"a string}") : unclosed string
The RJSONIO package behaves slightly differently. Rather than throwing an error, it silently returns a NULL value.
RJSONIO::fromJSON('{"x":"a string}')
# $x
# NULL