grep, awk, sed, cut

awk

$ awk '/search_pattern/ { action_to_take_on_matches; another_action; }' file_to_parse

• The first column can be referenced by $1 for instance. The entire line can by referenced by $0.

• Awk Internal Variables and Expanded Format:

  • FILENAME: References the current input file.
  • FNR: References the number of the current record relative to the current input file. For instance, if you have two input files, this would tell you the record number of each file instead of as a total.
  • FS: The current field separator used to denote each field in a record. By default, this is set to whitespace.
  • NF: The number of fields in the current record.
  • NR: The number of the current record.
  • OFS: The field separator for the outputted data. By default, this is set to whitespace.
  • ORS: The record separator for the outputted data. By default, this is a newline character.
  • RS: The record separator used to distinguish separate records in the input file. By default, this is a newline character.

• There are also optional BEGIN and END blocks that can contain commands to execute before and after the file processing, respectively.

  • awk 'BEGIN { action; } /search/ { action; } END { action; }' input_file

• awk '$2 ~ /^sa/' favorite_food.txt

  • ^ character tells awk to limit its searches to the beginning of the field.
  • field_num ~ part specifies that it should only pay attention to the field_num column.

tail

Print the last 10 lines of a file by default.

$ tail filename.txt

Print N number of lines from the file named filename.txt

$ tail -n N filename.txt

View the content of the file in real time using tail -f. This is useful to view the log files, that keeps growing. The command can be terminated using CTRL-C.

$ tail -f log-file

less

less is very efficient while viewing huge log files, as it doesn’t need to load the full file while opening.

$ less huge-log-file.log

One you open a file using less command, following two keys are very helpful.

CTRL+F – forward one window CTRL+B – backward one window

cat

Can be used to merge files in a directory cat * > merged-file

export

• export propagates the variable to subprocesses.

For example, if you did

FOO=bar then a subprocess that checked for FOO wouldn't find the variable whereas

export FOO=bar would allow the subprocess to find it.

But if FOO has already been defined as an environment variable, then FOO=bar will modify the value of that environment variable.

For example:

FOO=one     # Not an environment variable
export FOO  # Now FOO is an environment variable
FOO=two     # Update the environment variable, so sub processes will see $FOO = "two"