Tag: awk

Create IETF QUIC pcap file

I couldn’t find some off-the-shelf QUIC pcap files which conform to IETF draft versions, so I decided to create them myself. Take Client Initial packet in draft-29 as an example:

(1) Copy following raw packet info into payload.txt:

c5ff00001d088394c8f03e5157080000 449e4a95245bfb66bc5f93032b7ddd89
fe0ff15d9c4f7050fccdb71c1cd80512 d4431643a53aafa1b0b518b44968b18b
......

Use awk command to create string from payload.txt:

$ awk '{for (i = 1; i <= NF; i++) {printf("\"%s\"\n", $i)}}' payload.txt
"c5ff00001d088394c8f03e5157080000"
"449e4a95245bfb66bc5f93032b7ddd89"
"fe0ff15d9c4f7050fccdb71c1cd80512"
"d4431643a53aafa1b0b518b44968b18b"
......

(2) Use a C program to generate payload array:

#include <stdio.h>

char payload[] =
"c5ff00001d088394c8f03e5157080000"
"449e4a95245bfb66bc5f93032b7ddd89"
......
;

int main(void) {
    size_t count = 0;
    printf("const uint8_t payload[] = {\n");
    for (size_t i = 0; i < strlen(payload); i += 2) {
        printf("0x%c%c,", payload[i], payload[i + 1]);
        if (++count == 8) {
            count = 0;
            printf("\n");
        } else {
            printf(" ");
        }
    }
        printf("};");
    return 0;
}

(3) Forge ethernet, IP and UDP headers and use libpcap APIs to generate pcap file. The code can be referred here.

Compare error prompts of different awk flavours

Awk requires the opening brace of an action must be on the same line as the pattern it accompanies. I compared error prompts of different awk flavours, and my OS is NetBSD.

(1) awk shipped in NetBSD:

# awk 'BEGIN
> {print "Hello Awk!"}'
awk: syntax error at source line 2
 context is
        BEGIN >>>
 <<<
awk: bailing out at source line 2

(2) nawk:

# nawk 'BEGIN
{print "Hello Awk!"}'
nawk: syntax error at source line 2
 context is
        BEGIN >>>
 <<<
nawk: bailing out at source line 2

(3) mawk:

# mawk 'BEGIN
{print "Hello Awk!"}'
mawk: line 1: syntax error at or near end of line

(4) gawk:

# gawk 'BEGIN
{print "Hello Awk!"}'
gawk: cmd. line:2: BEGIN blocks must have an action part

IMHO, gawk gives more clear clues for this error.

Reassemble packets for pcap file

In TCP protocol, because MSS limitation, sometimes one endpoint needs to split one TCP packet into multiple packets and send them. Today, I met a case which requires to reassemble them into one.

Firstly, I used Wireshark to “Hex Dump” first need-reassemble packet:

0000   18 cf 24 4c 71 4b 54 89 98 76 b8 30 08 00 45 00
......

Modify the length in IP header, append remaining TCP payload, then used colrm to remove offset:

# colrm 1 4 < data > data.txt

Used awk to prepend 0x and append , for every value:

awk '{ for(i = 1; i <= NF; i++) {$i="0x"$i","} print}' data.txt

Added the variable definition for array:

const u_char new_packet_4[] = {
    0x18, 0xcf, ......
    .......
}

Lastly, write a small program to insert new packet 4 and remove original packet 4 and 5, and code is here (Don’t forget to modify the header of packet 4).

How to count the line number of a file?

How to count the line number of a file? It seems an easy question for Unix guys: using wc command. Let’s see an example:

# cat test
aaaa
bbbb
cccc
# wc -l test
3 test

It displays right. But wait a minute, let’s check the wc -l option meaning:

# wc --help
......
-l, --lines            print the newline counts
......

wc -l just print the newline counts, so it increases newline count as the line number. Using hexdump to see the content of test:

# hexdump -C test
00000000  61 61 61 61 0a 62 62 62  62 0a 63 63 63 63 0a     |aaaa.bbbb.cccc.|
0000000f

No problem, test contains 3 newlines. But now, the question is here, how about the last line doesn’t contain newline?

Let me do the following experiment:

# echo -n "a" > test
# hexdump -C test
00000000  61                                                |a|
00000001
# wc -l test
0 test

Now the test file only contains 1 character, and no newline. This time, wc -l thinks there is no line in test file.

Based on the previous discussion, maybe awk is a better tool when using to count line number of file. Just one statement:

# awk 'END {print NR}' test
1

It outputs right line number.