I can see tshark/wireshark has a flag to display only packets with checksum errors (tcp.checksum.status == "Unverified"). I tried checking if there is any similar flag in tcpdump, I couldn't find it. Is there a flag/option available with tcpdump? If so any one knows what it is? Thanks.
As you say, tcpdump only offers the filters specified here. This does not include an option to filter checksum.
However, you can easily filter checksum incorrect packets using grep:
sudo tcpdump -i eth0 -vvv tcp | grep incorrect
Otherwise your best option is to use tshark.
i am working on a project where we need to collect the bitorrent infohash id running in our small ISP network. using port mirroring we can pass the all wan traffic to a server and run tcpdump tools or any other tool to find the infohash id download by bitorrent client. for example
tcpflow -p -c -i eth1 tcp | grep -oE '(GET) .* HTTP/1.[01].*'
this code is showing result like this
GET /announce?info_hash=N%a1%94%17%2c%11%aa%90%9c%0a%1a0%9d%b2%cfy%08A%03%16&peer_id=-BT7950-%f1%a2%d8%8fO%d7%f9%bc%f1%28%15%26&port=19211&uploaded=55918592&downloaded=0&left=0&corrupt=0&key=21594C0B&numwant=200&compact=1&no_peer_id=1 HTTP/1.1
now we need to capture only infohash and store it to a log or mysql database
can you please tell me which tool can do thing like this
Depending on how rigorous you want to be you'll have to decode the following protocol layers:
TCP, assemble packets of a flow. you're already doing that with tcpflow. tshark - wireshark's CLI - could do that too.
HTTP, extract the value of the GET header. A simple regex would do the job here.
URI, extracting the query string
application/x-www-form-urlencoded, info_hash key value pair extraction and handling of percent-encoding
For the last two steps I would look for tools or libraries in your programming language of choice to handle them.
I write some expect scripts by connecting to a remote host through serial connection.
My problem is that the output of the spawned process (enabled with log_user 1) contains DOS-style endings (each line being terminated with ^M when reading logs in VIM).
I normally run dos2unix on all logs at the end of the expect session, to get rid of them. Can the conversion be done in real time?
It turned out the problem is really simple.
What I am doing with my scripts is calling them like below:
expect script.exp > mylog
As told in the description, mylog contains ^M line-endings when opened in Vim or using cat -v mylog.
To get rid of them in real-time, I just call now:
expect script.exp | tr -d '\r' > mylog
How can i use tcpdump to capture Ethernet frames and display any frame sent or received by the local PC with one of the UDP, ARP, and ICMP protocols.
I was trying this command:
sudo tcpdump -e udp or arp or icmp
but, i thinks it's wrong.
I can give you an example, how you can capture enthernet frame from your localhost.
sudo tcpdump -i lo -nnvvvexxXXKS -s0
for capturing the frame we used "exxXX"
Do use tcpdump -e. Here's an example of the output:
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
22:36:02.408697 02:42:ac:11:00:02 (oui Unknown) > 02:42:ac:11:00:03 (oui Unknown), ethertype IPv4 (0x0800), length 74: client.60546 > yahoo.com.80: Flags [S], seq 1673384407, win 64240, options [mss 1460,sackOK,TS val 2181456358 ecr 0,nop,wscale 7], length 0
In this example, you can see frame fields such as the MAC addresses (e.g. 02:42:ac:11:00:03) and the frame type (e.g. ethertype IPv4 0x0800).
From the manpage:
If the '-e' option is given, the link level header is printed out. On Ethernets, the source and destination addresses, protocol, and packet length are printed.
On FDDI networks, the '-e' option causes tcpdump to print the `frame control' field, the source and destination addresses, and the packet length. (The `frame control' field governs the interpretation of the rest of the packet. Normal packets (such as those containing IP datagrams) are `async' packets, with a priority value between 0 and 7; for example, `async4'. Such packets are assumed to contain an 802.2 Logical Link Control (LLC) packet; the LLC header is printed if it is not an ISO datagram or a so-called SNAP packet.
On Token Ring networks, the '-e' option causes tcpdump to print the `access control' and `frame control' fields, the source and destination addresses, and the packet length. As on FDDI networks, packets are assumed to contain an LLC packet. Regardless of whether the '-e' option is specified or not, the source routing information is printed for source-routed packets.
On 802.11 networks, the '-e' option causes tcpdump to print the `frame control' fields, all of the addresses in the 802.11 header, and the packet length. As on FDDI netâworks, packets are assumed to contain an LLC packet.
First of all, you are interested in packets, not frames. Frames are a layer below packets and only chip manufacturers are concerned with them. Second, you must specify your interface with the -i switch or promiscuous mode won't be even activated for you to see everything - if that's what you want.
I have set up a system on my Raspberry Pi to record some TCPDUMP data. This system works under a light workload, but for some unknown reason, doesn't work under my "heavy" traffic (27 relevant packets per second).
Under the last heavy traffic system I tried to record, my monitor.log file had 35,200 rows that only contained the last 16 minutes worth of data (judging by the timestamps). My filter.log also only goes back 16 minutes worth. There should be something like 1 million rows.
Could anyone advise on how to find the possible bug, bottle-necks, dropped pipe data, etc?
RC.LOCAL:
java -jar filter.jar > filter.log 2>&1 &
bash ./monitor &
MONITOR:
TCPDUMP -l | SED | tee monitor.log | tee myFIFO
You may try an utility like iptraf to monitor traffic.
try sed with stream option (unbuffered) ( -u on aix and --unbuffered on GNU sed) so sed does not wait for an EOF or assimilate (like a >> file from a discontinu stream)