I would like to pipe the tcpdump output to the stdin of a program of my choice.
But, I don't know what is the delimeter between each packet captured by the tcpdump.
I have gone through the manpage but have not found any setting to set a custom delimiter between each packet.
I would like to have the following format to be piped to the program of my choice:
Src_IP_of_Packet, data
Where the data is raw (not in ASCII and/or HEX format).
i.e. the data that I get using the following command.
tcpdump udp port 2112 -i eth0 -s0 -w-
With the above command, I am getting only the data but not the source IP.
Moreover, each packet seems to contain a newline character due to which each line is treated to be a new packet by my program because my program by default takes a newline to be the delimiter between each packet.
There is no delimiter and no way to greatly modify the output of tcpdump beyond making the output more verbose. If you are seeking to parse the data, it is best to view each line of text as a single packet, though adding one or more -v options will modify that display, resulting in variable numbers of lines for each packet.
If you would like to control which fields in the packet headers are displayed and add/modify delimiters, you will want to either look at tshark, which is a part of Wireshark, or interact with the packets through a scripting language; in this case, Python using Scapy is a great choice.
Related
Got some issues with Youtube API, it doesnt show åäö in titles. Urltitle script does, after an eggdrop recompile. So any suggestions how to make 2nd script use utf8 or so?
There are two key problems here:
Ensuring that the data from the data source is understood correctly by Tcl.
Ensuring that the correct data known by Tcl is reported to the channel in a way that it can digest.
Hopefully the first is already working; the http package and the various database access packages mostly get that right (except when the data source tells lies, as can happen occasionally with real-world data). You can test this by doing:
set msg ""
foreach char [split $data ""] { # For each UNICODE character...
append msg [format %04x [scan $char %c]]; # The hex code for the char
}
putserv "PRIVMSG $chan :$msg"
For example, this would generate a message like this for data of åäö€:
00e500e400f620ac
If that's working (it's the hard part to solve if it isn't), all you've got to do is ensure that the data actually goes correctly to the channel. This might be as simple as doing this (if the Tcl side of the channel is in binary mode):
putserv "PRIVMSG $chan :[encoding convertto utf-8 $data]"
In theory, the Tcl channel that putserv writes on could do the conversion for you, but getting that part right is tricky when there's unknown code between your code and the actual channel.
It's also possible that the IRC server expects the data in a different encoding such as iso8859-15. There isn't a universal rule there, alas.
I have a program read from stdin and process it. ( like "tee /some/file" )
This program wait stdin end to exit itself.
If I spawn it from Expect, after I send many content, how to send an "EOF" to the program?
there is a close command in Expect, but it will also send a SIGHUP, and can not expect program output anymore.
Expect works (on non-Windows) by using a virtual terminal which the spawned program runs within. This means that you can do things by sending character sequences to simulate keys. In particular, the EOF control sequence is done with Ctrl+D, which becomes the character U+000004. The terminal processes this to turn it into a true EOF.
There's a few ways to write it, depending on which escape sequence you prefer, but one of these will work:
# Hexadecimal-encoded escape
send \x04
# Octal-encoded escape
send \004
# UNICODE escape (also hexadecimal)
send \u0004
# Generate by a command
send [format "%c" 4]
When Expect is using Tcl 8.6, these all generate the same bytecode so use whichever you prefer.
How do you process a Chrome Native Messaging API-call with a bash script?
I succeeded in doing it with python with this example
Sure I can call bash from the python code with subprocess, but is it possible to skip python and process the message in bash directly?
The problematic part is reading the JSON serialized message into a variable. The message is serialized using JSON, UTF-8 encoded and is preceded with 32-bit message length in native byte order through stdin.
echo $* only outputs:
chrome-extension://knldjmfmopnpolahpmmgbagdohdnhkik/
Also something like
read
echo $REPLY
doesn't output anything. No sign of the JSON message. Python uses struct.unpack for this. Can that be done in bash?
I suggest to not use (bash) shell scripts as a native messaging host, because bash is too limited to be useful.
read without any parameters reads a whole line before terminating, while the native messaging protocol specifies that the first four bytes specify the length of the following message (in native byte order).
Bash is a terrible tool for processing binary data. An improved version of your read command would specify the -n N parameter to stop reading after N characters (note: not bytes) and -r to remove some processing. E.g. the following would store the first four characters in a variable called var_prefix:
IFS= read -rn 4 var_prefix
Even if you assume that this stores the first four bytes in the variable (it does not!), then you have to convert the bytes to an integer. Did I already mention that bash automatically drops all NUL bytes? This characteristics makes Bash utterly worthless for being a fully capable native messaging host.
You could cope with this shortcoming by ignoring the first few bytes, and start parsing the result when you spot a { character, the beginning of the JSON-formatted request. After this, you have to read all input until the end of the input is found. You need a JSON parser that stops reading input when it encounters the end of the JSON string. Good luck with writing that.
Generating output is a easier, just use echo -n or printf.
Here is a minimal example that assumes that the input ends with a }, reads it (without processing) and replies with a result. Although this demo works, I strongly recommend to not use bash, but a richer (scripting) language such as Python or C++.
#!/bin/bash
# Loop forever, to deal with chrome.runtime.connectNative
while IFS= read -r -n1 c; do
# Read the first message
# Assuming that the message ALWAYS ends with a },
# with no }s in the string. Adopt this piece of code if needed.
if [ "$c" != '}' ] ; then
continue
fi
message='{"message": "Hello world!"}'
# Calculate the byte size of the string.
# NOTE: This assumes that byte length is identical to the string length!
# Do not use multibyte (unicode) characters, escape them instead, e.g.
# message='"Some unicode character:\u1234"'
messagelen=${#message}
# Convert to an integer in native byte order.
# If you see an error message in Chrome's stdout with
# "Native Messaging host tried sending a message that is ... bytes long.",
# then just swap the order, i.e. messagelen1 <-> messagelen4 and
# messagelen2 <-> messagelen3
messagelen1=$(( ($messagelen ) & 0xFF ))
messagelen2=$(( ($messagelen >> 8) & 0xFF ))
messagelen3=$(( ($messagelen >> 16) & 0xFF ))
messagelen4=$(( ($messagelen >> 24) & 0xFF ))
# Print the message byte length followed by the actual message.
printf "$(printf '\\x%x\\x%x\\x%x\\x%x' \
$messagelen1 $messagelen2 $messagelen3 $messagelen4)%s" "$message"
done
I am using the terminal in a chromebook to ssh into a remote server. When I run a MySQL (5.6) select query, sometimes one of the fields will return nonsense unicode (when the field should return an email address) and change the MySQL prompt from:
mysql>
to
└≤⎽─┌>
and whatever text I type is converted into weird unicode. The problem persists even after I exit MySQL
One of the values in your database happened to have the sequence of bytes 0x1B, 0x28, 0x30 (ESC ) 0) in it. When you did the query, MySQL printed this byte sequence directly to your console. You can reproduce the effect by typing from python:
>>> print '\x1B\x28\x30'
Consoles use control characters (in particular 0x1B, ESC) as a way to allow applications to control aspects of the console other than pure text, such as colours and cursor movements. This behaviour is inherited from the old dumb-terminal devices that they are pretending to be (which is why they are also known as terminal emulators), along with some weirder tricks that we probably don't need any more. One of those is to switch permanently between different character sets (considered encodings, now, but this long predates Unicode).
One of those alternative character sets is the DEC Special Graphics Character Set which it looks like you have here. In this character set the byte 0x6D, usually used in ASCII for m, comes out as the graphical character └.
You could in principle reset your terminal to normal ASCII by printing a byte sequence 0x1B, 0x28, 0x42 (ESC ) B), but this tends to be a pain to arrange when your console is displaying rubbish.
There are potentially other ways your console can become confused; it's not, in general safe to print arbitrary binary data to the console. There even used to be nastier things you could do with the console by faking keyboard input, which made this a security problem, but today it's just an annoyance factor.
However, one wouldn't normally expect to have any control codes in an e-mail address field. I suggest the application using the database should be doing some validation on the input it receives, and dropping or blocking all control codes (other than potentially newlines where necessary).
As a quick hack to clean this field for the specific case of the ESC character, you could do something like:
UPDATE things SET email=REPLACE(email, CHAR(0x1B), '');
I actually am generating an MS Excel file with the currencies and if you see the file I generated (tinyurl.com/currencytestxls), opening it in the text editor shows the correct symbol but somehow, MS Excel does not display the symbol. I am guessing there is some issue with the encoding. Any thoughts?
Here is my tcl code to generate the symbol:
set yen_val [format %c 165]
Firstly, this does produce a Yen symbol (I put format string in double quotes here just for clarity with the formatting):
format "%c" 165
You can then pass it around just fine. The problem is likely to come when you try to output it; when Tcl writes a string to the outside world (with the possible exception of the terminal on Windows, as that's tricky) it encodes that string into a definite byte sequence. The default encoding is the one reported by:
encoding system
But you can see what it is and change it for any channel (if you pass in the new name):
fconfigure $theChannel -encoding $theEncoding
For example, on my system (which uses UTF-8, which can handle any character):
% fconfigure stdout -encoding
utf-8
% puts [format %c 165]
¥
If you use an encoding that cannot represent a particular character, the replacement character for that encoding is used instead. For many encodings, that's a “?”. When you are sending data to another program (including to a web server or to a browser over the internet) it is vital that both sides agree on what the encoding of the data is. Sometimes this agreement is by convention (e.g., the system encoding), sometimes it is defined by the protocol (HTTP headers have this clearly defined), and sometimes this is done by explicitly transferred metadata (HTTP content).
If you're writing a CSV file to be ingested by Excel, use either the “unicode” or the “utf-8” encoding and make sure you put the byte-order mark in correctly. Tcl doesn't write BOMs automatically (because it's the wrong thing to do in some cases). To write a BOM, do this as the first thing when you start writing the file:
puts -nonewline $channel "\ufeff"