I am performing a number of trace routes to different IP's throughout the course of 1 week. Ive got a script that performs a set of trace routes and writes and appends them to the same .log file.
This file is obviously now quite large as I'm performing trace route 3 times a day on 6 targets for a week. Im trying to write a simple program that will convert my log files into CSV format for analysis in Excel.
Before each trace route runs it prints ''--- START ---'' and finishes with ''--- END ---''. See the following example:
--- START ---
Mon Mar 12 22:45:05 GMT 2012
traceroute to xxxxxxxx (xxxxxx), 30 hops max, 60 byte packets
1 xxxxxxx (xxxxxxx) 1.085 ms 1.662 ms 2.244 ms
2 xxxxxx (xxxxxx) 0.792 ms 0.782 ms 0.772 ms
3 xxxxxx (xxxxxx) 8.545 ms 9.170 ms 9.644 ms
4 etc
5 etc
--- END ---
--- START ---
Mon Mar 12 22:45:05 GMT 2012
traceroute to xxxxxx (xxxxx), 30 hops max, 60 byte packets
1 xxxxxxx (xxxxxxx) 0.925 ms 1.318 ms 1.954 ms
2 xxxxx (xxxxxx) 0.345 ms 0.438 ms 0.496 ms
3 xxxxxxx (xxxxxx) 0.830 ms 2.553 ms 0.809 ms
4 etc
5 etc
--- END ---
I was going to use the START and END to delimit and separate each trace route from one another. I also need to take the total number of jumps that each trace routes makes, that being the last number on the line before ''--- END ---".
If anyone could help me out it would be great. I need something that will run through each trace route, separating them. And then showing the number of hops each trace route makes... Im currently using MATLAB.
Cheers.
The best way to solve your problem is using regex. Just find those start and end tags and for each match make the necessary processing :)
Related
I am trying to read dtc codes using following procedure:
1 - send 0101 which means get the number of the dtc codes
receive - 41 01 82 07 61 01 which means I have 2 trouble codes
2 - send 0x03 to receive codes and
receive - 43 01 33 03 01 00 00 which are the 2 codes : P0133 and P0301
Using a diagnostic service tester I get 5 codes on the same car: P0133, P0301 (the ones I get also) and P0303, P0300 and C1513
How can I see the other 3 codes using my OBD?
Here are pictures with the tester:
You might have multiple OBD2-conform ECUs in your car. To access these, use broadcast addressing or query each and every one separately by adjusting your arbitration ids.
That said, some readers may also show pending (0x07) or permanent (0x0A) DTCs.
I'm developing an interface to a VeriFone VX terminal. Although, this is really a general EMV question. Our processor has a zero floor limit, so it will always be sent online. However, in case it ever changes, how do you know (what tags) if the transaction was approved or declined offline? Or, in other words, how do you know to go online or not?
how do you know (what tags) if the transaction was approved or declined offline? Or, in other words, how do you know to go online or not?
The terminal has to decides either to proceed the transaction offline, to go online or to reject the transaction. Here terminal send a command (AC) to the card and response of this command helps terminal to decide the action next followed.
Decision making is depend on three fields -
1) - Issuer Action Code
2) - Terminal Action Code
3) - TVR
IAC, TAC and TVR have the same structure. For more to know this data you can see EMV BOOK 3
IAC Usage Example-
suppose IAC-ONLINE (TAG - 9F0F) = 08 00 00 00 00 ,
here byte 1 bit 4 is on i.e. offline DDA Failed ,
Here Issuer want to go online if offline DDA Failed.
when terminal perform DDA and it fails, it set corresponding bit in TVR
that means TVR says- offline DDA is failed for this card.
now terminal check IAC online and found DDA_Failed bit is on and same on in TVR, here terminal decision would be to go online and then it send a Gen AC command to card with p1 = 80 ( ARQC - Online authorisation requested).
Coding of P1 as below
Ex- Gen AC command
C: 80 AE 80 00 other data
R: SW1/SW2=9000 (Normal processing: No error) Lr=32
77 1E 9F 27 01 80 9F 36 02 02 13 9F 26 08 2D F3
83 3C 61 85 5B EA 9F 10 07 06 84 23 00 31 02 08
.
Now decision is made by card, Terminal get card decision in the response of Gen AC command. Card return tag 9F27 - Cryptogram Information Data. here card return 80 i.e. cards wants transaction to go Online.
Really your question is important and you need to read more spec for clarity on this topic. Please checks EMV BOOKs, for more in this topic. also can read - Terminal action analysis or Card Action analysis
Assuming you're using VeriFone's VIPA API, then the first 'Continue Transaction' command (GenAC1) returns tags wrapped in a TLV template (or 'constructed' TLV tag). The value of this template determines the result:
E3: Locally authorized
E4: Requires online authorization
AFAIK (in vanilla EMV) the tag Cryptogram Information Data ('9F27') returned during 1st GENERATE AC should serve this purpose.
See EMV Book 3, Table 14.
Beware, that this tag contains the decision of the card, so you won't see the cryptogram type the kernel required.
I've got some good experience with pdflib when it comes to speed of pdf-generation, even for large files. I was excpecting the same speeds from princexml as for pdflib, as both run natively on my linux server (they're not just php-classes). When generating a 1 page pdf with text and graphics, I see a 4 second time lapse between begin and loading of the document in the log file? Is this normal? The conversion itself doesn't seem to be long...
Mon Apr 16 19:17:30 2012: ---- begin
Mon Apr 16 19:17:34 2012: Loading document...
Mon Apr 16 19:17:34 2012: Converting document...
Mon Apr 16 19:17:34 2012: finished: success
Mon Apr 16 19:17:34 2012: ---- end
Are there network connections involved in your setup? Is there DNS name resolution involved? If yes, try to use IP addresses instead of hostnames and try again...
I'm currently developing an practice application in node.js. This applications consists of a JSON REST web service which allows two services.
Insert log (a PUT request to /log, with the message to log)
Last 100 logs (a GET request to /log, that returns the latest 100 logs)
The current stack is formed by a node.js server that has the application logic and a mongodb database that takes care of the persistence. To offer the JSON REST web services I'm using the node-restify module.
I'm currently executing some stress tests using apache bench (using 5000 requests with a concurrency of 10) and get the following results:
Execute stress tests
1) Insert log
Requests per second: 754.80 [#/sec] (mean)
2) Last 100 logs
Requests per second: 110.37 [#/sec] (mean)
I'm surprised of the difference there is in performance, the query I'm executing uses an index. Interestingly enough it seems that the JSON output generation seems to get all the time on deeper tests I have performed.
Can node applications be profiled in detail?
Is this behaviour normal? Retrieving data takes so much more than inserting data?
EDIT:
Full test information
1) Insert log
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Server Software: log-server
Server Hostname: localhost
Server Port: 3010
Document Path: /log
Document Length: 0 bytes
Concurrency Level: 10
Time taken for tests: 6.502 seconds
Complete requests: 5000
Failed requests: 0
Write errors: 0
Total transferred: 2240634 bytes
Total PUT: 935000
HTML transferred: 0 bytes
Requests per second: 768.99 [#/sec] (mean)
Time per request: 13.004 [ms] (mean)
Time per request: 1.300 [ms] (mean, across all concurrent requests)
Transfer rate: 336.53 [Kbytes/sec] received
140.43 kb/s sent
476.96 kb/s total
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.1 0 3
Processing: 6 13 3.9 12 39
Waiting: 6 12 3.9 11 39
Total: 6 13 3.9 12 39
Percentage of the requests served within a certain time (ms)
50% 12
66% 12
75% 12
80% 13
90% 15
95% 24
98% 26
99% 30
100% 39 (longest request)
2) Last 100 logs
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Server Software: log-server
Server Hostname: localhost
Server Port: 3010
Document Path: /log
Document Length: 4601 bytes
Concurrency Level: 10
Time taken for tests: 46.528 seconds
Complete requests: 5000
Failed requests: 0
Write errors: 0
Total transferred: 25620233 bytes
HTML transferred: 23005000 bytes
Requests per second: 107.46 [#/sec] (mean)
Time per request: 93.057 [ms] (mean)
Time per request: 9.306 [ms] (mean, across all concurrent requests)
Transfer rate: 537.73 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.1 0 1
Processing: 28 93 16.4 92 166
Waiting: 26 85 18.0 86 161
Total: 29 93 16.4 92 166
Percentage of the requests served within a certain time (ms)
50% 92
66% 97
75% 101
80% 104
90% 113
95% 121
98% 131
99% 137
100% 166 (longest request)
Retrieving data from the database
To query the database I use the mongoosejs module. The log schema is defined as:
{
date: { type: Date, 'default': Date.now, index: true },
message: String
}
and the query I execute is the following:
Log.find({}, ['message']).sort('date', -1).limit(100)
Can node applications be profiled in detail?
Yes. Use node --prof app.js to create a v8.log, then use linux-tick-processor, mac-tick-processor or windows-tick-processor.bat (in deps/v8/tools in the node src directory) to interpret the log. You have to build d8 in deps/v8 to be able to run the tick processor.
Here's how I do it on my machine:
apt-get install scons
cd ~/development/external/node-0.6.12/deps/v8
scons arch=x64 d8
cd ~/development/projects/foo
node --prof app.js
D8_PATH=~/development/external/node-0.6.12/deps/v8 ~/development/external/node-0.6.12/deps/v8/tools/linux-tick-processor > profile.log
There are also a few tools to make this easier, including node-profiler and v8-profiler (with node-inspector).
Regarding your other question, I would like some more information on how you fetch your data from Mongo, and what the data looks like (I agree with beny23 that it looks like a suspiciously low amount of data).
I strongly suggest taking a look at the DTrace support of Restify. It will likely become your best friend when profiling.
http://mcavage.github.com/node-restify/#DTrace
I am fetching data from MySql Server into R using RODBC.
So in one column of the database is a character vector
SELECT MAX(CHAR_LENGTH(column)) FROM reqtable;
RETURNS 26566
Now I will show you an example how I am running into the problem
`library(RODBC)
con <- odbcConnect("mysqlcon")
rslts <- as.numeric(sqlQuery(con,
"SELECT CHAR_LENGTH(column) FROM reqtable LIMIT 10",
as.is=TRUE)[,1])
`
returns
> rslts
[1] 62 31 17 103 30 741 28 73 25 357
where as
rslts <- nchar(as.character(sqlQuery(con,
"SELECT column FROM reqtable LIMIT 10",
as.is=TRUE)[,1]))
returns
> rslts
[1] 62 31 17 103 30 255 28 73 25 255
So strings with length > 255 is getting truncated at 255. Is there a way I can get the full string.
Thanks
The PostgreSQL ODBC driver has a variable called MaxLongVarcharSize that I have found set to 8190 by default (I've used it both on Windows and Ubuntu). It is possible that the MySQL ODBC driver has a similar variable set to 255.
You could try to use another db driver such as JDBC. In my experience this has sometimes solved the problem.
Also, try the RMySQL package (current binaries need to be compiled. if you do compile them yourself, request you to please share with the community)
Probably the source of the RODBC package "could" provide insights into the default length limitations if any. (I haven't looked at it yet, but I will soon and post an update here)
Another possibility why the retrieved number of characters might be limited is a 'sanity' check restriction to 65535 bytes in the RODBC package itself -- as mentioned here.