I have an old vb6 program which queries an access 2000 database. I have a fairly long query which looks something like this:
Select * from table where key in ( 0, 1, 2, 3, 4, 5, 6, 7, 11, 12, 13, 14, 15, 19, 20, 21, 24, 27, 29, 30, 35, 38, 39, 40, 42, 43, 44, 46, 47, 49, 50, 53, 56, 59, 60, 61, 63, 64, 65, 66, 67, 68, 72, 76, 80, 84, 86, 89, 90, 91, 93, 94, 98, 99, 10041, 10042, 10045, 10046, 10047, 10049, 10057, 10060, 10089, 32200, 32202, 32203, 32204, 32205, 32207, 32214, 32245, 32303, 32314, 32403, 32405, 32414, 32415, 32503, 32703, 32803, 32903, 33003, 33014, 33102, 33103, 33303, 33403, 33405, 33601, 33603, 33604, 33614, 33705, 33714, 33901, 33903, 33914, 34001, 34105, 34114, 34203, 34303, 34401, 34501, 34601, 34603, 34604, 34605, 34803, 41001, 41005, 41007, 41013, 42001, 42005, 42007, 42013, 43001, 43002, 44001, 44007, 46001, 46007, 99999, 9999999)
However, when I look at the RecordSource of the data object, it seems that the query is being truncated to this (which is obviously not syntactically valid and throws an error):
Select * from table where key in ( 0, 1, 2, 3, 4, 5, 6, 7, 11, 12, 13, 14, 15, 19, 20, 21, 24, 27, 29, 30, 35, 38, 39, 40, 42, 43, 44, 46, 47, 49, 50, 53, 56, 59, 60, 61, 63, 64, 65, 66, 67, 68, 72, 76, 80, 84, 86, 89, 90, 91, 93, 94, 98, 99, 100
My data source looks like this:
Begin VB.Data dtaList
Caption = "dtaList"
Connect = "Access 2000;"
DatabaseName = ""
DefaultCursorType= 0 'DefaultCursor
DefaultType = 2 'UseODBC
Exclusive = 0 'False
Height = 345
Left = 960
Options = 0
ReadOnly = 0 'False
RecordsetType = 1 'Dynaset
RecordSource = ""
Top = 4440
Visible = 0 'False
Width = 2295
End
I've tried running the full query in the access database itself which works fine.
Is this a limitation in the VB.Data object, or is there some other explanation? Is there any way I can get around this issue?
Unfortunately I am unable to upgrade to a newer version of access.
The truncated version of the SQL statement you posted is 246 characters long, so it appears that something along the line is limiting the length of the SQL string to somewhere around 255 characters. As you have discovered by pasting the query into Access itself, the actual size limit of an Access query string is much larger (around 64,000 characters, I believe).
I remember running across a similar issue years ago but my problem was an INSERT statement that was writing some rather long strings to the database. The workaround in that case was to use a parameter query (which I realize, in hindsight, that I should have been using anyway). It greatly shortened the length of the SQL string because the parameters were passed separately. Unfortunately that workaround probably wouldn't help you because even if you dynamically created a parameterized version of the query it wouldn't be all that much shorter than the current SQL string.
Another workaround would be to write all of those numbers for the IN clause as rows in a temporary table named something like [inValues], and then use the query
SELECT [table].*
FROM
[table]
INNER JOIN
[inValues]
ON [table].[key] = [inValues].[key]
Related
What I did
Stored a UUID as BINARY(16) in NodeJS using
const uuid = Buffer.from('myEditedUuid');
(A followup to How do I fetch binary columns from MySQL in Rust?)
What I want to do
I want to fetch said UUID using Rust https://docs.rs/mysql/20.0.0/mysql/.
I am currently using Vec<u8> to gain said UUID:
#[derive(Debug, PartialEq, Eq, Serialize)]
pub struct Policy {
sub: String,
contents: Option<String>,
}
#[derive(Debug, PartialEq, Eq, Serialize)]
pub struct RawPolicy {
sub: Option<Vec<u8>>,
contents: Option<String>,
}
// fetch policies themselves
let policies: Vec<RawPolicy> = connection.query_map("SELECT sub, contents FROM policy", |(sub, contents)| {
RawPolicy { sub, contents }
},)?;
// convert uuid to string
let processed = policies.into_iter().map(|policy| {
let sub = policy.sub.unwrap();
let sub_string = String::from_utf8(sub).unwrap().to_string();
Policy {
sub: sub_string,
contents: policy.contents,
}
}).collect();
What my problem is
In Node, I would receive a Buffer from said database and use something like uuidBUffer.toString('utf8');
So in Rust, I try to use String::from_utf8(), but said Vec does not seem to be a valid utf8-vec:
panicked at 'called `Result::unwrap()` on an `Err` value: FromUtf8Error { bytes: [17, 234, 79, 61, 99, 181, 10, 240, 164, 224, 103, 175, 134, 6, 72, 71], error: Utf8Error { valid_up_to: 1, error_len: Some(1) } }'
My question is
Is Using Vec correct way of fetching BINARY-Columns and if so, how do I convert them back to a string?
Edit1:
Node seems to use Base 16 to Convert A string to a Buffer (Buffer.from('abcd') => <Buffer 61 62 63 64>).
Fetching my parsed UUID in Rust made With Buffer.from() gives me Vec<u8> [17, 234, 79, 61, 99, 181, 10, 240, 164, 224, 103, 175, 134, 6, 72, 71] which thows said utf8-Error.
Vec does not seem to be allowed by MySQL in Rust.
Solution is simple:
You need to convert the BINARY to hex at you database Query or you code. So either try Using the HEX-Crate https://docs.rs/hex/0.4.2/hex/ or rewrite your Query:
Rewriting The Query
let policies: Vec<RawPolicy> = connection.query_map("SELECT hex(sub), contents FROM policy", |(sub, contents)| {
RawPolicy { sub, contents }
},)?;
Converts the sub to hex numbers. Now the resulting Vec can be converted using
let sub = policy.sub.unwrap();
let sub_string = String::from_utf8(sub).unwrap();
from_utf8_lossy can be used
let input = [17, 234, 79, 61, 99, 181, 10, 240, 164, 224, 103, 175, 134, 6, 72, 71];
let output = String::from_utf8_lossy(&input); // "\u{11}�O=c�\n��g��\u{6}HG"
Invalid characters will be replaced by �
The output "\u{11}�O=c�\n��g��\u{6}HG" is the same as the nodejs output "\u0011�O=c�\n��g��\u0006HG".
Unless this string is to be send to a javascript runtime, it should be kept that way.
But if this string is to be send to a javascript runtime (browser or nodejs), then the unicode point notations\u{x} should be substituted to their equivalent notation in javascript
playground
from_ut16_lossy can be used as well
If some of the previous � are not utf-8 encoded but utf-16, they will be converted, if not the same � will be used to render them.
let input:&[u16] = &vec![17, 234, 79, 61, 99, 181, 10, 240, 164, 224, 103, 175, 134, 6, 72, 71];
println!("{}", String::from_utf16_lossy(input))
playground
I have a CSV file that look something like below: i.e. not in Prolog format
james,facebook,intel,samsung
rebecca,intel,samsung,facebook
Ian,samsung,facebook,intel
I am trying to write a Prolog predicate that reads the file and returns a list that looks like
[[james,facebook,intel,samsung],[rebecca,intel,samsung,facebook],[Ian,samsung,facebook,intel]]
to be used further in other predicates.
I am still a beginner and have found some good information from SO and modified them to see if I can get it but I`m stuck because I only generate a list that looks like this
[[(james,facebook,intel,samsung)],[(rebecca,intel,samsung,facebook)],[(Ian,samsung,facebook,intel)]]
which means when I call the head of the inner lists I get (james,facebook,intel,samsung) and not james.
Here is the code being used :- (seen on SO and modified)
stream_representations(Input,Lines) :-
read_line_to_codes(Input,Line),
( Line == end_of_file
-> Lines = []
; atom_codes(FinalLine, Line),
term_to_atom(LineTerm,FinalLine),
Lines = [[LineTerm] | FurtherLines],
stream_representations(Input,FurtherLines)
).
main(Lines) :-
open('file.txt', read, Input),
stream_representations(Input, Lines),
close(Input).
The problem lies with term_to_atom(LineTerm,FinalLine).
First we read a line of the CSV file into a list of character codes in
read_line_to_codes(Input,Line).
Let's simulate input with atom_codes/2:
?- atom_codes('james,facebook,intel,samsung',Line).
Line = [106, 97, 109, 101, 115, 44, 102, 97, 99|...].
Then we recompose the original atom read in into FinalLine (this seems wasteful, there must be a way to hoover up a line into an atom directly)
?- atom_codes('james,facebook,intel,samsung',Line),
atom_codes(FinalLine, Line).
Line = [106, 97, 109, 101, 115, 44, 102, 97, 99|...],
FinalLine = 'james,facebook,intel,samsung'.
The we try to map this atom in FinalLine into a term, LineTerm, using term_to_atom/2
?- atom_codes('james,facebook,intel,samsung',Line),
atom_codes(FinalLine, Line),
term_to_atom(LineTerm,FinalLine).
Line = [106, 97, 109, 101, 115, 44, 102, 97, 99|...],
FinalLine = 'james,facebook,intel,samsung',
LineTerm = (james, facebook, intel, samsung).
You see the problem here: LineTerm is not quite a list, but a nested term using the functor , to separate elements:
?- atom_codes('james,facebook,intel,samsung',Line),
atom_codes(FinalLine, Line),
term_to_atom(LineTerm,FinalLine),
write_canonical(LineTerm).
','(james,','(facebook,','(intel,samsung)))
Line = [106, 97, 109, 101, 115, 44, 102, 97, 99|...],
FinalLine = 'james,facebook,intel,samsung',
LineTerm = (james, facebook, intel, samsung).
This ','(james,','(facebook,','(intel,samsung))) term will thus also be in the final result, just written differently: (james,facebook,intel,samsung) and packed into a list:
[(james,facebook,intel,samsung)]
You do not want this term, you want a list. You could use atomic_list_concat/2 to create a new atom that can be read as a list:
?- atom_codes('james,facebook,intel,samsung',Line),
atom_codes(FinalLine, Line),
atomic_list_concat(['[',FinalLine,']'],ListyAtom),
term_to_atom(LineTerm,ListyAtom),
LineTerm = [V1,V2,V3,V4].
Line = [106, 97, 109, 101, 115, 44, 102, 97, 99|...],
FinalLine = 'james,facebook,intel,samsung',
ListyAtom = '[james,facebook,intel,samsung]',
LineTerm = [james, facebook, intel, samsung],
V1 = james,
V2 = facebook,
V3 = intel,
V4 = samsung.
But that's rather barbaric.
We must do this whole processing in fewer steps:
Read a line of comma-separated strings on input.
Transform this into a list of either atoms or strings directly.
DCGs seem like the correct solution. Maybe someone can add a two-liner.
I have some code that I have been porting from 2.7 to 3.6/3.7. Most of the unit tests, which have a pretty good coverage, already execute successfully under 3.x. But I have yet to fully commit to switching over to 3.x for development.
I recently noticed, when running black - the code formatter that it chokes if my code would not compile under 3.x, with a message about 3.6 AST-based parsing failing.
Is black a reliable indicator of 3.x-readiness, at least at the syntax level? I know that 2to3 is the tool to use. And I know that for example, it would not catch differences in the standard library (basestring disappearing, StringIO.StringIO becoming io.StringIO, etc...).
but it seems nice that a code formatter could incidentally help out as well.
very basic sample, invalid syntax for 3.x:
print "a", 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21
gives:
error: cannot format test_black.py:
cannot use --safe with this file; failed to parse source file with
Python 3.6's builtin AST.
Re-run with --fast or stop using deprecated Python 2 syntax.
AST error message: Missing parentheses in call to 'print'.
Did you mean print("a", 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21)? (<unknown>, line 1)
All done! 💥 💔 💥
1 file failed to reformat.
fix the syntax to 3.x and it works.
If I do the right thing, and add parenthesis print ("a", 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21), then all's well:
reformatted test_black.py
All done! ✨ 🍰 ✨
1 file reformatted.
I've been having a little trouble figuring out how to calculate the time complexity for merge-sorting this series of numbers: 44, 112, 178, 48, 10, 22, 28, 186, 66, 86, 128, 82, 168. The numbers should end up in ascending order.
I'm not sure if this will help or not, but here's a diagram showing the merge-sorting function being run on a different set of numbers: http://upload.wikimedia.org/wikipedia/commons/thumb/e/e6/Merge_sort_algorithm_diagram.svg/300px-Merge_sort_algorithm_diagram.svg.png
I have some data files with content
a1 b1 c1 d1
a1 b2 c2 d2
...
[blank line]
a2 b1 c1 d1
a2 b2 c2 d2
...
I plot this with gnuplot using
splot 'file' u 1:2:3:4 w pm3d.
Now, I want to use a binary file. I created the file with Fortran using unformatted stream-access (direct or sequential access did not work directly). By using gnuplot with
splot 'file' binary format='%float%float%float%float' u 1:2:3
I get a normal 3D-plot. However, the pm3d-command does not work as I don't have the blank lines in the binary file. I get the error message:
>splot 'file' binary format='%float%float%float%float' u 1:2:3:4 w pm3d
Warning: Single isoline (scan) is not enough for a pm3d plot.
Hint: Missing blank lines in the data file? See 'help pm3d' and FAQ.
According to the demo script in http://gnuplot.sourceforge.net/demo/image2.html, I have to specify the record length (which I still don't understand right). However, using this script from the demo page and the command with pm3d obtains the same error message:
splot 'scatter2.bin' binary record=30:30:29:26 u 1:2:3 w pm3d
So how is it possible to plot this four dimensional data from a binary file correctly?
Edit: Thanks, mgilson. Now it works fine. Just for the record: My fortran code-snippet:
open(unit=83,file=fname,action='write',status='replace',access='stream',form='unformatted')
a= 0.d0
b= 0.d0
do i=1,200
do j=1,100
write(83)real(a),real(b),c(i,j),d(i,j)
b = b + db
end do
a = a + da
b = 0.d0
end do
close(83)
The gnuplot commands:
set pm3d map
set contour
set cntrparam levels 20
set cntrparam bspline
unset clabel
splot 'fname' binary record=(100,-1) format='%float' u 1:2:3:4 t 'd as pm3d-projection, c as contour'
Great question, and thanks for posting it. This is a corner of gnuplot I hadn't spent much time with before. First, I need to generate a little test data -- I used python, but you could use fortran just as easily:
Note that my input array (b) is just a 10x10 array. The first two "columns" in the datafile are just the index (i,j), but you could use anything.
>>> import numpy as np
>>> a = np.arange(10)
>>> b = a[None,:]+a[:,None]
>>> b
array([[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
[ 2, 3, 4, 5, 6, 7, 8, 9, 10, 11],
[ 3, 4, 5, 6, 7, 8, 9, 10, 11, 12],
[ 4, 5, 6, 7, 8, 9, 10, 11, 12, 13],
[ 5, 6, 7, 8, 9, 10, 11, 12, 13, 14],
[ 6, 7, 8, 9, 10, 11, 12, 13, 14, 15],
[ 7, 8, 9, 10, 11, 12, 13, 14, 15, 16],
[ 8, 9, 10, 11, 12, 13, 14, 15, 16, 17],
[ 9, 10, 11, 12, 13, 14, 15, 16, 17, 18]])
>>> with open('foo.dat','wb') as foo:
... for (i,j),dat in np.ndenumerate(b):
... s = struct.pack('4f',i,j,dat,dat)
... foo.write(s)
...
So here I just write 4-floating point values to the file for each data-point. Again, this is what you've already done using fortran. Now for plotting it:
splot 'foo.dat' binary record=(10,-1) format='%float' u 1:2:3:4 w pm3d
I believe that this specifies that each "scan" is a "record". Since I know that each scan will be 10 floats long, that becomes the first index in the record list. The -1 indicates that gnuplot should keep reading records until it finds the end of the file.