LINUX - bash history & more - not by me

THIS TOP PART IS BY ME BUT THE BOTTOM PART IS NOT


MAKE HISTORY RECORDING MORE RELIABLE WITH DATE AND TIME + MORE
BEST HISTORY ALTERATIONS (PUT INTO .bashrc FOR USERS THAT YOU WANT THIS / ALL USERS WOULD BE BEST - just append these to the end of your .bashrc and you will be happy)
shopt -s histappend
HISTFILESIZE=1000000
HISTSIZE=1000000
HISTCONTROL=ignoredups
HISTTIMEFORMAT='%F %T '
shopt -s cmdhist
PROMPT_COMMAND='history -a'

WHAT THOSE DO IS LISTED BELOW:
Basically append history to history file @ end of every session, increase the history size for the file and the session, ignore duplicate enteries and only list them once uniquely, save with timeformat (when you enable this feature all things in history before this command will have the date of when you enabled the feature - which is quite nice), also convert multiline commands into single lines in history, and finally save each command into the history immediately.

MORE INFO ON THOSE BELOW

TO CLEAR HISTORY - BASIC (when PROMPT_COMMAND is not set to 'history -a')

How to check if PROMPT_COMMAND is set? echo $PROMPT_COMMAND if you see that it is set to 'history -a' then stop what your doing and read the next section.


Which says to run:

rm -f $HISTFILE
unset HISTFILE
Then exit out of the shell with "exit" or just close out of the shell

TO CLEAR HISTORY WHEN PROMPT_COMMAND='history -a'

Now those CLEAR HISTORY BASIC steps alone would be fine if we didnt have PROMPT_COMMAND='history -a', but we do (as long as you followed the MAKE HISTORY MORE RELIABLE STEPS) so we need one more command before all of these to unset it.

unset PROMPT_COMMAND
rm -f $HISTFILE
unset HISTFILE
Then exit out of the shell with "exit" or just close out of the shell

Dont forget to set PROMPT_COMMAND back to 'history -a' with the following command - but only after you log back in:
PROMPT_COMMAND='history -a'

So what if we didnt unset PROMPT_COMMAND, what would happen? Well after you log back in and check history you would see "rm -f $HISTFILE" so it would be blatently obvious someone tried to mess with the history.

BEING UNCONSPICOUS WITH BASH HISTORY DELETING
ANOTHER TIP:
If your doing this to hide your steps, always to the following steps
unset PROMPT_COMMAND
rm -f $HISTFILE
unset HISTFILE
And never log back in and type "history" to confirm...
If you need to confirm with "history" - like I do, you will need to do these steps twice, once to try it and confirm everthing was deleted with history (the only command you will see that was type would be "history"), so then repeat the steps again -- if you dont who ever is checking will see the first command is "history" which is also semi conspicous
So the whole process would look something like this:
...random conspicous work...
...check to see if PROMPT_COMMAND is set to history -a...
echo $PROMPT_COMMAND
...if its set do this, and continue with the rest of the work...
unset PROMPT_COMMAND
..if its not set then just do the rest, and dont unset anything (although it doesnt hurt to unset a variable thats not set already, but I mean it could be set to something else and we dont want to distrupt that)...
rm -f $HISTFILE
unset HISTFILE
exit
...relog in...
history
...you will see nothing except one history command...
...so repeat the steps...
unset PROMPT_COMMAND
rm -f $HISTFILE
unset HISTFILE
exit
...dont log back in... (or if you do need to set PROMP_COMMAND back up, just log back in to set PROMPT_COMMAND back up and then exit like this: relog in then type PROMPT_COMMAND='history -a'; exit;)




This is for my own reference - the source is here: http://blog.sanctum.geek.nz - this person wrote these genious articles I need quick access too.

This next article is not written by me, I copied it from (I hope its okay, whoever you are I am giving you full credit as I just admitted to not be the author of the next brilliant article): http://blog.sanctum.geek.nz 



tter Bash history

By default, the Bash shell keeps the history of your most recent session in the .bash_history file, and the commands you’ve issued in your current session are also available with a history call. These defaults are useful for keeping track of what you’ve been up to in the shell on any given machine, but with disks much larger and faster than they were when Bash was designed, a little tweaking in your .bashrc file can record history more permanently, consistently, and usefully.

Append history instead of rewriting it

You should start by setting the histappend option, which will mean that when you close a session, your history will be appended to the .bash_history file rather than overwriting what’s in there.

shopt -s histappend

Allow a larger history file

The default maximum number of commands saved into the .bash_history file is a rather meager 500. If you want to keep history further back than a few weeks or so, you may as well bump this up by explicitly setting $HISTSIZE to a much larger number in your .bashrc. We can do the same thing with the$HISTFILESIZE variable.

HISTFILESIZE=1000000
HISTSIZE=1000000

The man page for Bash says that HISTFILESIZE can be unset to stop truncation entirely, but unfortunately this doesn’t work in .bashrc files due to the order in which variables are set; it’s therefore more straightforward to simply set it to a very large number.

If you’re on a machine with resource constraints, it might be a good idea to occasionally archive old.bash_history files to speed up login and reduce memory footprint.

Don’t store specific lines

You can prevent commands that start with a space from going into history by setting $HISTCONTROL toignorespace. You can also ignore duplicate commands, for example repeated du calls to watch a file grow, by adding ignoredups. There’s a shorthand to set both in ignoreboth.

HISTCONTROL=ignoreboth

You might also want to remove the use of certain commands from your history, whether for privacy or readability reasons. This can be done with the $HISTIGNORE variable. It’s common to use this to exclude lscalls, job control builtins like bg and fg, and calls to history itself:

HISTIGNORE='ls:bg:fg:history'

Record timestamps

If you set $HISTTIMEFORMAT to something useful, Bash will record the timestamp of each command in its history. In this variable you can specify the format in which you want this timestamp displayed when viewed with history. I find the full date and time to be useful, because it can be sorted easily and works well with tools like cut and awk.

HISTTIMEFORMAT='%F %T '

Use one command per line

To make your .bash_history file a little easier to parse, you can force commands that you entered on more than one line to be adjusted to fit on only one with the cmdhist option:

shopt -s cmdhist

Store history immediately

By default, Bash only records a session to the .bash_history file on disk when the session terminates. This means that if you crash or your session terminates improperly, you lose the history up to that point. You can fix this by recording each line of history as you issue it, through the $PROMPT_COMMAND variable:

PROMPT_COMMAND='history -a'
Bash history expansion
Posted on August 16, 2012

Setting the Bash option histexpand allows some convenient typing shortcuts using Bash history expansion. The option can be set with either of these:

$ set -H
$ set -o histexpand

It’s likely that this option is already set for all interactive shells, as it’s on by default. The manual, man bash, describes these features as follows:

-H  Enable ! style history substitution. This option is on
    by default when the shell is interactive.

You may have come across this before, perhaps to your annoyance, in the following error message that comes up whenever ! is used in a double-quoted string, or without being escaped with a backslash:

$ echo "Hi, this is Tom!"
bash: !": event not found

If you don’t want the feature and thereby make ! into a normal character, it can be disabled with either of these:

$ set +H
$ set +o histexpand

History expansion is actually a very old feature of shells, having been available in csh before Bash usage became common.

This article is a good followup to Better Bash history, which among other things explains how to include dates and times in history output, as these examples do.

Basic history expansion

Perhaps the best known and most useful of these expansions is using !! to refer to the previous command. This allows repeating commands quickly, perhaps to monitor the progress of a long process, such as disk space being freed while deleting a large file:

$ rm big_file &
[1] 23608
$ du -sh .
3.9G    .
$ !!
du -sh .
3.3G    .

It can also be useful to specify the full filesystem path to programs that aren’t in your $PATH:

$ hdparm
-bash: hdparm: command not found
$ /sbin/!!
/sbin/hdparm

In each case, note that the command itself is printed as expanded, and then run to print the output on the following line.

History by absolute index

However, !! is actually a specific example of a more general form of history expansion. For example, you can supply the history item number of a specific command to repeat it, after looking it up with history:

$ history | grep expand
 3951  2012-08-16 15:58:53  set -o histexpand
$ !3951
set -o histexpand

You needn’t enter the !3951 on a line by itself; it can be included as any part of the command, for example to add a prefix like sudo:

$ sudo !3850

If you include the escape string \! as part of your Bash prompt, you can include the current command number in the prompt before the command, making repeating commands by index a lot easier as long as they’re still visible on the screen.

History by relative index

It’s also possible to refer to commands relative to the current command. To subtitute the second-to-last command, we can type !-2. For example, to check whether truncating a file with sed worked correctly:

$ wc -l bigfile.txt
267 bigfile.txt
$ sed -i '11,$d' bigfile.txt
$ !-2
wc -l bigfile.txt
10 bigfile.txt

This works further back into history, with !-3!-4, and so on.

Expanding for historical arguments

In each of the above cases, we’re substituting for the whole command line. There are also ways to get specific tokens, or words, from the command if we want that. To get the first argument of a particular command in the history, use the !^ token:

$ touch a.txt b.txt c.txt
$ ls !^
ls a.txt
a.txt

To get the last argument, add !$:

$ touch a.txt b.txt c.txt
$ ls !$
ls c.txt
c.txt

To get all arguments (but not the command itself), use !*:

$ touch a.txt b.txt c.txt
$ ls !*
ls a.txt b.txt c.txt
a.txt  b.txt  c.txt

This last one is particularly handy when performing several operations on a group of files; we could run duand wc over them to get their size and character count, and then perhaps decide to delete them based on the output:

$ du a.txt b.txt c.txt
4164    a.txt
5184    b.txt
8356    c.txt
$ wc !*
wc a.txt b.txt c.txt
16689    94038  4250112 a.txt
20749   117100  5294592 b.txt
33190   188557  8539136 c.txt
70628   399695 18083840 total
$ rm !*
rm a.txt b.txt c.txt

These work not just for the preceding command in history, but also absolute and relative command numbers:

$ history 3
 3989  2012-08-16 16:30:59  wc -l b.txt
 3990  2012-08-16 16:31:05  du -sh c.txt
 3991  2012-08-16 16:31:12  history 3
$ echo !3989^
echo -l
-l
$ echo !3990$
echo c.txt
c.txt
$ echo !-1*
echo c.txt
c.txt

More generally, you can use the syntax !n:w to refer to any specific argument in a history item by number. In this case, the first word, usually a command or builtin, is word 0:

$ history | grep bash
 4073  2012-08-16 20:24:53  man bash
$ !4073:0
man
What manual page do you want?
$ !4073:1
bash

You can even select ranges of words by separating their indices with a hyphen:

$ history | grep apt-get
 3663  2012-08-15 17:01:30  sudo apt-get install gnome
$ !3663:0-1 purge !3663:3
sudo apt-get purge gnome

You can include ^ and $ as start and endpoints for these ranges, too. 3* is a shorthand for 3-$, meaning “all arguments from the third to the last.”

Expanding history by string

You can also refer to a previous command in the history that starts with a specific string with the syntax!string:

$ !echo
echo c.txt
c.txt
$ !history
history 3
 4011  2012-08-16 16:38:28  rm a.txt b.txt c.txt
 4012  2012-08-16 16:42:48  echo c.txt
 4013  2012-08-16 16:42:51  history 3

If you want to match any part of the command line, not just the start, you can use !?string?:

$ !?bash?
man bash

Be careful when using these, if you use them at all. By default it will run the most recent command matching the string immediately, with no prompting, so it might be a problem if it doesn’t match the command you expect.

Checking history expansions before running

If you’re paranoid about this, Bash allows you to audit the command as expanded before you enter it, with thehistverify option:

$ shopt -s histverify
$ !rm
$ rm a.txt b.txt c.txt

This option works for any history expansion, and may be a good choice for more cautious administrators. It’s a good thing to add to one’s .bashrc if so.

If you don’t need this set all the time, but you do have reservations at some point about running a history command, you can arrange to print the command without running it by adding a :p suffix:

$ !rm:p
rm important-file

In this instance, the command was expanded, but thankfully not actually run.

Substituting strings in history expansions

To get really in-depth, you can also perform substitutions on arbitrary commands from the history with!!:gs/pattern/replacement/. This is getting pretty baroque even for Bash, but it’s possible you may find it useful at some point:

$ !!:gs/txt/mp3/
rm a.mp3 b.mp3 c.mp3

If you only want to replace the first occurrence, you can omit the g:

$ !!:s/txt/mp3/
rm a.mp3 b.txt c.txt

Stripping leading directories or trailing files

If you want to chop a filename off a long argument to work with the directory, you can do this by adding an :hsuffix, kind of like a dirname call in Perl:

$ du -sh /home/tom/work/doc.txt
$ cd !$:h
cd /home/tom/work

To do the opposite, like a basename call in Perl, use :t:

$ ls /home/tom/work/doc.txt
$ document=!$:t
document=doc.txt

Stripping extensions or base names

A bit more esoteric, but still possibly useful; to strip a file’s extension, use :r:

$ vi /home/tom/work/doc.txt
$ stripext=!$:r
stripext=/home/tom/work/doc

To do the opposite, to get only the extension, use :e:

$ vi /home/tom/work/doc.txt
$ extonly=!$:e
extonly=.txt

Quoting history

If you’re performing substitution not to execute a command or fragment but to use it as a string, it’s likely you’ll want to quote it. For example, if you’ve just found through experiment and trial and error an idealffmpeg command line to accomplish some task, you might want to save it for later use by writing it to a script:

$ ffmpeg -f alsa -ac 2 -i hw:0,0 -f x11grab -r 30 -s 1600x900 \
> -i :0.0+1600,0 -acodec pcm_s16le -vcodec libx264 -preset ultrafast \
> -crf 0 -threads 0 "$(date +%Y%m%d%H%M%S)".mkv 

To make sure all the escaping is done correctly, you can write the command into the file with the :qmodifier:

$ echo '#!/usr/bin/env bash' >ffmpeg.sh
$ echo !ffmpeg:q >>ffmpeg.sh

In this case, this will prevent Bash from executing the command expansion "$(date ... )", instead writing it literally to the file as desired. If you build a lot of complex commands interactively that you later write to scripts once completed, this feature is really helpful and saves a lot of cutting and pasting.

Thanks to commenter Mihai Maruseac for pointing out a bug in the examples.

Command line editing

Posted on January 30, 2012

By default, the Bash shell uses GNU Readline, which provides a set of Emacs-friendly key bindings that are a pretty workable way to edit long and complicated commands. If you learn a little about the chords available to you in editing Bash commands through this library, and combine that with a little old-school command-line magic, you can work at high speed with decent accuracy quite easily. Alternatively, if you don’t like the Emacs-style keybindings and would prefer to stick to a vi-friendly model, you can use set -o vi to edit your command line directly with vi keybindings.

However, if you’re building a particularly complex string of commands involving a lot of pipes, escapes, and redirections, it often turns out to be handy to actually load them into your favourite editor, to give you full facility to edit them in any way you wish. Bash provides a method for this in the form of its Ctrl+X, Ctrl+E binding.

To use it, you can type anything at the command prompt (including nothing at all) and press Ctrl+X, Ctrl+E to bring up your EDITOR, be it Vim, Emacs, or Nano, with the contents of the command line there to edit. As soon as you save and quit, the command will be run as stated, and will be entered into the command history as if you typed it out on the shell directly.

There’s also a handy built-in Bash shortcut, fc (short for “fix command”) to open the previous command in your editor, allow you to edit it, and then run it automatically when you quit. This is particularly useful if you’ve made a small mistake in a complex line of shell code.

If this happens to bring up the wrong editor, perhaps because your choice doesn’t match that of the system administrator, you can set your personal preference of editor like so:

$ export EDITOR=/usr/bin/vim

You can confirm this is working by checking your environment variables:

$ env

One downside of this method is that without special setup within your editor, you lose some of the benefits of things like tab completion. Fortunately it only takes a little creative mapping to make this work in Vim, taking advantage of the Ctrl+X, Ctrl+F file completion that’s already built in. You could even bind that straight to the Tab key if you don’t otherwise use it.

:inoremap <Tab> <C-X><C-F>
High-speed Bash
Posted on January 24, 2012

One of my favourite technical presentations I’ve read online has been Hal Pomeranz’s Unix Command-Line Kung Fu, a catalogue of shortcuts and efficient methods of doing very clever things with the Bash shell. None of these are grand arcane secrets, but they’re things that are often forgotten in the course of daily admin work, when you find yourself typing something you needn’t, or pressing up repeatedly to find something you wrote for which you could simply search your command history.

I highly recommend reading the whole thing, as I think even the most experienced shell users will find there are useful tidbits in there that would make their lives easier and their time with the shell more productive, beyond simpler things like tab completion. Here, I’ll recap two of the things I thought were the most simple and useful items in the presentation for general shell usage, and see if I can add a little value to them with reference to the Bash manual.

History with Ctrl+R

For many shell users, finding a command in history means either pressing the up arrow key repeatedly, or perhaps piping a history call through grep. It turns out there’s a much nicer way to do this, using Bash’s built-in history searching functionality; if you press Ctrl+R and start typing a search pattern, the most recent command matching that pattern will automatically be inserted on your current line, at which point you can adapt it as you need, or simply press Enter to run it again. You can keep pressing Ctrl+R to move further back in your history to the next-most recent match. On my shell, if I search through my history for git, I can pull up what I typed for a previous commit:

(reverse-i-search)`git': git commit -am "Pulled up-to-date colors."

This functionality isn’t actually exclusive to Bash; you can establish a history search function in quite a few tools that use GNU Readline, including the MySQL client command line.

You can search forward through history in the same way with Ctrl+S, but it’s likely you’ll have to fix up a couple of terminal annoyances first.

Additionally, if like me you’re a Vim user and you don’t really like having to reach for the arrow keys, or if you’re on a terminal where those keys are broken for whatever reason, you can browse back and forth within your command history with Ctrl+P (previous) and Ctrl+N (next). These are just a few of the Emacs-style shortcuts that GNU Readline provides; check here for a more complete list.

Repeating commands with !!

The last command you ran in Bash can be abbreviated on the next line with two exclamation marks:

$ echo "Testing."
Testing.
$ !!
Testing.

You can use this to simply repeat a command over and over again, although for that you really should be using watch, but more interestingly it turns out this is very handy for building complex pipes in stages. Suppose you were building a pipeline to digest some data generated from a program like netstat, perhaps to determine the top 10 IP addresses that are holding open the most connections to a server. You might be able to build a pipeline like this:

# netstat -ant
# !! | awk '{print $5}'
# !! | sort
# !! | uniq -c
# !! | sort -rn
# !! | sed 10q

Similarly, you can repeat the last argument from the previous command line using !$, which is useful if you’re doing a set of operations on one file, such as checking it out via RCS, editing it, and checking it back in:

$ co -l file.txt
$ vim !$
$ ci -u !$

Or if you happen to want to work on a set of arguments, you can repeat all of the arguments from the previous command using !*:

$ touch a.txt b.txt c.txt
$ rm !*

When you remember to user these three together, they can save you a lot of typing, and will really increase your accuracy because you won’t be at risk of mistyping any of the commands or arguments. Naturally, however, it pays to be careful what you’re running through rm!


Typo correction in Bash
Posted on March 23, 2012

If you make a typo in a long command line in Bash, instead of typing it all out again, you can either recall it from your history, or use caret notation to repeat the command with an appropriate substitution. This looks like the following:

# sudo apache2ctl restrat
Action 'restrat' failed.
The Apache error log may have more information.
# ^strat^start
sudo apache2ctl restart

The string after the first caret is the text to match, and the one after the second string is the text with which it should be replaced. This provides a convenient method of not only quickly correcting typos, but to change small parts of the command line in general quickly:

$ touch filea.txt
$ ^filea^fileb
touch fileb.txt
$ ^fileb^filec
touch filec.txt

For the particular case of correcting small errors in long paths for cd calls, it’s helpful to use the cdspelloption for Bash, which I discuss in my earlier article on smarter directory navigation.

Bash prompts

Posted on July 9, 2012

You can tell a lot about a shell user by looking at their prompt. Most shell users will use whatever the system’s default prompt is for their entire career. Under many Linux distributions, this prompt includes the username, the hostname, and the current working directory, along with a $ sigil for regular users, and a #for root.

tom@sanctum:~$

This format works well for most users, and it’s so common that it’s often used in tutorials to represent the user’s prompt in lieu of the single-character standard $ and #. It shows up as the basis for many of the custom prompt setups you’ll find online.

Some users may have made the modest steps of perhaps adding some brackets around the prompt, or adding the time or date:

[10:11][tom@sanctum:~]$

Still others make the prompt into a grand multi-line report of their shell’s state, or indeed their entire system, full of sound and fury and signifying way too much:

-(23:38:57)-(4 users)-(0.00, 0.02, 0.05)-
-(Linux sanctum 3.2.0-3-amd64 x86_64 GNU/Linux)-
[tom@sanctum:/dev/pts/2:/home/tom]{2}!?[MAIL]$

Then there are the BSD users and other minimalist contrarians who can’t abide anything more than a single character cluttering their screens:

$

Getting your prompt to work right isn’t simply a cosmetic thing, however. If done right and in a style consistent with how you work, it can provide valuable feedback on how the system is reacting to what you’re doing with it, and also provide you with visual cues that could save you a lot of confusion — or even prevent your making costly mistakes.

I’ll be working in Bash. Many of these principles work well for Zsh as well, but Zsh users might want to read Steve Losh’s article on his prompt setup first to get an idea of the syntactical differences.

Why is this important?

One of the primary differences between terminals and windowed applications is that there tends to be much less emphasis on showing metadata about the running program and its environment. For the most part, to get information about what’s going on, you need to request it with calls like pwdwhoamireadlinkenv,jobs, and echo $?.

The habit of keeping metadata out of the terminal like this unless requested may appeal to a sense of minimalism, but like a lot of things in the command line world, it’s partly the olden days holding us back. When mainframes and other dinosaurs roamed the earth, terminal screens were small, and a long or complex prompt amounted to a waste of space and cycles (or even paper). In our futuristic 64-bit world, where tiny CRTs are a thing of the past and we think nothing of throwing gigabytes of RAM at a single process, this is much less of a problem.

Customising the prompt doesn’t have many “best practices” to it like a lot of things in the shell; there isn’t really a right way to do it. Some users will insist it’s valuable to have the path to the terminal device before every prompt, which I think is crazy as I almost never need that information, and when I do, I can get it with a call to tty. Others will suggest that including the working directory makes the length of the prompt too variable and distracting, whereas I would struggle to work without it. It’s worth the effort to design a prompt that reflects your working style, and includes the information that’s consistently important to you on the system.

So, don’t feel like you’re wasting time setting up your prompt to get it just right, and don’t be so readily impressed by people with gargantuan prompts in illegible rainbow colors that they copied off some no-name wiki. If you use Bash a lot, you’ll be staring at your prompt for several hours a day. You may as well spend an hour or two to make it useful, or at least easier on the eyes.

The basics

The primary Bash prompt is stored in the variable PS1. Its main function is to signal to the user that the shell is done with its latest foreground task, and is ready to accept input. PS1 takes the form of a string that can contain escape sequences for certain common prompt elements like usernames and hostnames, and which is evaluated at printing time for escape sequences, variables, and command or function calls within it.

You can inspect your current prompt definition with echo. If you’re using Linux, it will probably look similar to this:

tom@sanctum:~$ echo $PS1
\u@\h:\w\$

PS1 works like any other Bash variable; you can assign it a new value, append to it, and apply substitutions to it. To add the text bash to the start of my prompt, I could do this:

tom@sanctum:~$ PS1="bash-$PS1"
bash-tom@sanctum:~$

There are other prompts — PS2 is used for “continued line” prompts, such as when you start writing a forloop and continue another part of it on a new line, or terminate your previous line with a backslash to denote the next line continues on from it. By default this prompt is often a right angle bracket >:

$ for f in a b c; do
>   do echo "$f";
> done

There are a couple more prompt strings which are much more rarely seen. PS3 is used by the selectstructure in shell scripts; PS4 is used while tracing output with set -x. Ramesh Natarajan breaks this down very well in his article on the Bash prompt. Personally, I only ever modify PS1, because I see the other prompts so rarely that the defaults work fine for me.

As PS1 is a property of interactive shells, it makes the most sense to put its definitions in $HOME/.bashrcon Linux or BSD systems. It’s likely there’s already code in there doing just that. For Mac OS X, you’ll likely need to put it into $HOME/.bash_profile instead.

Escapes

Bash will perform some substitutions for you in the first pass of its evaluation of your prompt string, replacingescape sequences with appropriate elements of information that are often useful in prompts. Below are the escape sequences as taken from the Bash man page:

\a     an ASCII bell character (07)
\d     the date in "Weekday Month Date" format (e.g., "Tue May 26")
\D{format}
       the format is passed to strftime(3) and the result is inserted into
       the prompt string; an empty format results in a locale-specific time
       representation. The braces are required
\e     an ASCII escape character (033)
\h     the hostname up to the first `.'
\H     the hostname
\j     the number of jobs currently managed by the shell
\l     the basename of the shell's terminal device name
\n     newline
\r     carriage return
\s     the name of the shell, the basename of $0 (the portion following
       the final slash)
\t     the current time in 24-hour HH:MM:SS format
\T     the current time in 12-hour HH:MM:SS format
\@     the current time in 12-hour am/pm format
\A     the current time in 24-hour HH:MM format
\u     the username of the current user
\v     the version of bash (e.g., 2.00)
\V     the release of bash, version + patch level (e.g., 2.00.0)
\w     the current working directory, with $HOME abbreviated with a tilde
       (uses the value of the PROMPT_DIRTRIM variable)
\W     the basename of the current working directory, with $HOME
       abbreviated with a tilde
\!     the history number of this command
\#     the command number of this command
\$     if the effective UID is 0, a #, otherwise a $
\nnn   the character corresponding to the octal number nnn
\\     a backslash
\[     begin a sequence of non-printing characters, which could be used to
       embed a terminal control sequence into the prompt
\]     end a sequence of non-printing characters

As an example, the default PS1 definition on many Linux systems uses four of these codes:

\u@\h:\w\$

The \! escape sequence, which expands to the history item number of the current command, is worth a look in particular. Including this in your prompt allows you to quickly refer to other commands you may have entered in your current session by history number with a ! prefix, without having to actually consulthistory:

$ PS1="(\!)\$"
(6705)$ echo "test"
test
(6706)$ !6705
echo "test"
test

Keep in mind that you can refer to history commands relatively rather than by absolute number. The previous command can be referenced by using !! or !-1, and the one before that by !-2, and so on, provided that you have left the histexpand option on.

Also of particular interest are the delimiters \[ and \]. You can use these to delimit sections of “non-printing characters”, typically things like terminal control characters to do things like moving the cursor around, starting to print in another color, or changing properties of the terminal emulator such as the title of the window. Placing these non-printing characters within these delimiters prevents the terminal from assuming an actual character has been printed, and hence correctly manages things like the user deleting characters on the command line.

Colors

There are two schools of thought on the use of color in prompts, very much a matter of opinion:

  • Focus should be on the output of commands, not on the prompt, and using color in the prompt makes it distracting, particularly when output from some programs may also be in color.
  • The prompt standing out from actual program output is a good thing, and judicious use of color conveys useful information rather than being purely decorative.

If you agree more with the first opinion than the second, you can probably just skip to the next section of this article.

Printing text in color in terminals is done with escape sequences, prefixed with the \e character as above. Within one escape sequence, the foreground, background, and any styles for the characters that follow can be set.

For example, to include only your username in red text as your prompt, you might use the following PS1definition:

PS1='\[\e[31m\]\u\[\e[0m\]'

Broken down, the opening sequence works as follows:

  • \[ -- Open a series of non-printing characters, in this case, an escape sequence.
  • \e -- An ASCII escape character, used to confer special meaning to the characters that follow it, normally to control the terminal rather than to enter any characters.
  • [31m -- Defines a display attribute, corresponding to foreground red, for the text to follow; opened with a [ character and terminated with an m.
  • \] — Close the series of non-printing characters.

The terminating sequence is very similar, except it uses the 0 display attribute to reset the text that follows back to the terminal defaults.

The valid display attributes for 16-color terminals are:

  • Style
    • 0 — Reset all attributes
    • 1 — Bright
    • 2 — Dim
    • 4 — Underscore
    • 5 — Blink
    • 7 — Reverse
    • 8 — Hidden
  • Foreground
    • 30 — Black
    • 31 — Red
    • 32 — Green
    • 33 — Yellow
    • 34 — Blue
    • 35 — Magenta
    • 36 — Cyan
    • 37 — White
  • Background
    • 40 — Black
    • 41 — Red
    • 42 — Green
    • 43 — Yellow
    • 44 — Blue
    • 45 — Magenta
    • 46 — Cyan
    • 47 — White

Note that blink may not work on a lot of terminals, often by design, as it tends to be unduly distracting. It’ll work on a linux console, though. Also keep in mind that for a lot of terminal emulators, by default “bright” colors are printed in bold.

More than one of these display attributes can be applied in one hit by separating them with semicolons. The below will give you underlined white text on a blue background, after first resetting any existing attributes:

\[\e[0;4;37;44m\]

It’s helpful to think of these awkward syntaxes in terms of opening and closing, rather like HTML tags, so that you open before and then reset after all the attributes of a section of styled text. This usually amounts to printing \[\e[0m\] after each such passage.

These sequences are annoying to read and to type, so it helps to put commonly used styles into variables:

color_red='\[\e[31m\]'
color_blue='\[\e[34m\]'
color_reset='\[\e[0m\]'

PS1=${color_red}'\u'${color_reset}'@'${color_blue}'\h'${color_reset}

A more robust method is to use the tput program to print the codes for you, and put the results into variables:

color_red=$(tput setaf 1)
color_blue=$(tput setaf 4)

This may be preferable if you use some esoteric terminal types, as it queries the terminfo or termcapdatabases to generate the appropriate escapes for your terminal.

The set of eight colors — or sixteen, depending on how you look at it — can be extended to 256 colors in most modern terminal emulators with a little extra setup. The escape sequences are not very different, but their range is vastly increased to accommodate the larger palette.

A useful application for prompt color can be a quick visual indication of the privileges available to the current user. The following code changes the prompt to green text for a normal user, red for root, and yellow for any other user attained via sudo:

color_root="$(tput setaf 1)"
color_user="$(tput setaf 2)"
color_sudo="$(tput setaf 3)"
color_reset="$(tput sgr0)"

if [[ "$EUID" -eq 0 ]]; then
    PS1="\\[$color_root\\]$PS1\\[$color_reset\\]"
elif [[ -n "$SUDO_USER" ]]; then
    PS1="\\[$color_sudo\\]$PS1\\[$color_reset\\]"
else
    PS1="\\[$color_user\\]$PS1\\[$color_reset\\]"
fi

Otherwise, you could simply use color to make different regions of your prompt more or less visually prominent.

Variables

Provided the promptvars option is enabled, you can reference Bash variables like $? to get the return value of the previous command in your prompt string. Some people find this valuable to show error codes in the prompt when the previous command given fails:

$ shopt -s promptvars
$ PS1='$? \$'
1 $ echo "test"
test
0 $ fail
-bash: fail: command not found
127 $

The usual environment variables like USER and HOME will work too, though it’s preferable to use the escape sequences above for those particular variables. Note that it’s important to quote the variable references as well, so that they’re evaluated at runtime, and not during the assignment itself.

Commands

If the information you want in your prompt isn’t covered by any of the escape codes, you can include the output of commands in your prompt as well, using command substitution with the syntax $(command):

$ PS1='[$(uptime) ]\$ '
[ 01:21:59 up 7 days, 13:04,  7 users, load average: 0.30, 0.40, 0.36 ]$

This requires the promptvars option to be set, the same way variables in the prompt do. Again, note that you need to use single quotes so that the command is run when the prompt is being formed, and not immediately as part of the assignment.

This can include pipes to format the data with tools like awk or cut, if you only need a certain part of the output:

$ PS1='[$(uptime | cut -d: -f5) ]\$ '
[ 0.36, 0.34, 0.34 ]$

For clarity, during prompt setup in .bashrc, it makes sense to use functions instead for reasonably complex commands:

prompt_load() {
    uptime | cut -d: -f5
}
PS1='[$(prompt_load) ]'${PS1}

Note that as a normal part of command substitution, trailing newlines are stripped from the output of the command, so here the output of uptime appears on a single line.

A common usage of this pattern is showing metadata about the current directory, particularly if it happens to be a version control repository or working copy; you can use this syntax with functions to show the type of version control system, the current branch, and whether any changes require committing. Working with Git, Mercurial, and Subversion most often, I include the relevant logic as part of my prompt function.

When appended to my PS1 string, $(prompt vcs) gives me prompts that look like the following when I’m in directories running under the appropriate VCS. The exclamation marks denote that there are uncommitted changes in the repositories.

[tom@conan:~/.dotfiles](git:master!)$
[tom@conan:~/Build/tmux](svn:trunk)$
[tom@conan:~/Build/vim](hg:default!)$

In general, where this really shines is adding pieces to your prompt conditionally, to make them collapsible. Certain sections of your prompt therefore only show up if they’re relevant. This snippet, for example, prints the number of jobs running in the background of the current interactive shell in curly brackets, but only if the count is non-zero:

prompt_jobs() {
    if [[ -n "$(jobs)" ]]; then
        printf '{%d}' "$(jobs | sed -n '$=')"
    fi
}

It’s important to make sure that none of what your prompt does takes too long to run; an unresponsive prompt can make your terminal sessions feel very clunky.

Note that you can also arrange to run a set of commands before the prompt is evaluated and printed, using the PROMPT_COMMAND. This tends to be a good place to put commands that don’t actually print anything, but that do need to be run before or after each command, such as operations working with history:

PROMPT_COMMAND='history -a'

Switching

If you have a very elaborate or perhaps even computationally expensive prompt, it may occasionally be necessary to turn it off to revert to a simpler one. I like to handle this by using functions, one of which sets up my usual prompt and is run by default, and another which changes the prompt to the minimal $ or #character so often used in terminal demonstrations. Something like the below works well:

prompt_on() {
    PS1=${color_prompt}'\u@\h:\w$(prompt_jobs)\$'${color_reset}' '
}
prompt_off() {
    PS1='\$'
}
prompt_on

You can then switch your prompt whenever you need to by typing prompt_on and prompt_off.

This can also be very useful if you want to copy text from your terminal into documentation, or into an IM or email message; it removes your distracting prompt from the text, where it would otherwise almost certainly differ from that of the user following your instructions. This is also occasionally helpful if your prompt does not work on a particular machine, or the machine is suffering a very high load average that means your prompt is too slow to load.

Further reading

Predefined prompt strings are all over the web, but the above will hopefully enable you to dissect what they’re actually doing more easily and design your own. To take a look at some examples, the relevant page on the Arch Wiki is a great start. Ramesh Natarajan over at The Geek Stuff has a great article with some examples as well, with the curious theme of making your prompt as well-equipped as Angelina Jolie.

Finally, please feel free to share your prompt setup in the comments (whether you’re a Bash user or not). It would be particularly welcome if you explain why certain sections of your prompt are so useful to you for your particular work.


256 colour terminals

Posted on February 23, 2012

Using 256 colours in terminals is well-supported in Linux distributions these days, and also in Windows terminal emulators like PuTTY. Using 256 colours is great for Vim colorschemes in particular, but also very useful for Tmux colouring or any other terminal application where a slightly wider colour space might be valuable. Be warned that once you get this going reliably, there’s no going back if you spend a lot of time in the terminal.

Xterm

To set this up for xterm or emulators that use xterm as the default value for $TERM, such asxfce4-terminal or gnome-terminal, it generally suffices to check the options for your terminal emulator to ensure that it will allow 256 colors, and then use the TERM string xterm-256color for it.

An earlier version of this post suggested changing the TERM definition in .bashrc, which is generally not a good idea, even if bounded with conditionals as my example was. You should always set the terminal string in the emulator itself if possible, if you do it at all.

Be aware that older systems may not have terminfo definitions for this terminal, but you can always copy them in using a private .terminfo directory if need be.

Tmux

To use 256 colours in Tmux, you should set the default terminal in .tmux.conf to be screen-256color:

set -g default-terminal "screen-256color"

This will allow you to use color definitions like colour231 in your status lines and other configurations. Again, this particular terminfo definition may not be present on older systems, so you should copy it into~/.terminfo/s/screen-256color on those systems if you want to use it everywhere.

GNU Screen

Similarly, to use 256 colours in GNU Screen, add the following to your .screenrc:

term screen-256color

Vim

With the applicable options from the above set, you should not need to change anything in Vim to be able to use 256-color colorschemes. If you’re wanting to write or update your own 256-colour compatible scheme, it should either begin with set t_Co=256, or more elegantly, check the value of the corresponding option value is &t_Co is 256 before trying to use any of the extra colour set.

The Vim Tips Wiki contains a detailed reference of the colour codes for schemes in 256-color terminals.




Bash command existence




Posted on Ma2

If you set environment variables like your EDITOR in your .bashrc file that refer to commands that you expect to be available on the system, it’s prudent to check that an appropriate command actually exists before making the setting.

A common way of approaching this is using which, in a syntax similar to the below, which sets EDITOR to the first executable instance of vi found in PATH, or blank if not found:

EDITOR="$(which vi)"

Because the behaviour of which can be unexpected, it’s better practice to use one of Bash’s builtins to do this, either commandtype, or hash. I prefer using hash, which searches PATH for a command of the given name, and loads it into Bash’s command hash table if found. An implementation like the below works well:

if hash vi &>/dev/null; then
    export EDITOR=vi
fi

This ignores any output from the hash call by redirecting it to the null device, and only defines the value forEDITOR if a matching command is found.

You can compact this syntax into one line:

hash vi &>/dev/null && export EDITOR=vi

Thanks to commenter Yu-Jie Lin for pointing out the above abbreviation.

Avoiding the arrow keys

Posted on August 8, 2012

For shell users, moving to the arrow keys on the keyboard is something of an antipattern, moving away from the home row so central to touch typists. It therefore helps to find ways to avoid using the arrow keys in order to maintain your flow.

Bash

The arrow keys in Bash are used to move back and forth on the current command line (left and right), and up and down through the command history (up and down). This leads to the old shell user’s maxim:

“Those that don’t remember history are doomed to press Up repeatedly.”

There are alternatives to both functions. To move left or right by one character on the command line without deleting characters already placed, we can use Ctrl-B and Ctrl-F.

However, to make things a bit faster, there are four other key combinations to move back and forth on a line that are worth learning too:

  • Alt-B — Move back a word
  • Alt-F — Move forward a word
  • Ctrl-A — Move to the start of the line
  • Ctrl-E — Move to the end of the line

Similarly, to move up and down through history, we can use Ctrl-P and Ctrl-N respectively. Don’t forget that rather than keep tapping one of these, you can search backward or forward through history with Ctrl-Rand Ctrl-S.

Whoops, I think I just taught you some Emacs.

Vim

To avoid the arrow keys in normal mode in Vim, use hjk, and l instead. This can take a little getting used to, but the benefits in terms of comfort for your hands and speed is easily worth it; using the arrow keys is one of the Vim anti-patterns.

If you’re asking “How do I avoid the arrow keys to move in insert mode?”, the answer is that you shouldn’t be moving in insert mode at all; it’s inefficient. When in insert mode you should be inserting text, and maybe using backspace or delete every now and then. To move through text, switching back to normal mode is vastly more efficient.

Moving in command mode is similar. If you need to move around on the command line, in most cases you should pull up the command window so that you can edit the command in normal mode, and then just tapEnter to run it.

In general, when you start thinking about moving through any kind of text, you should reflexively hit Esc orCtrl-[ to go into normal mode, in order to take advantage of a whole keyboard’s worth of navigation shortcuts.

SelectionFile type iconFile nameDescriptionSizeRevisionTimeUser
Comments