12 years ago

A hero left a comment.

It Works For Me™. Try running strace -s9999 -f -o tmux.strace tmux -c true and post the output of grep -C5 'tmux\.conf' tmux.strace. –Gilles 'SO- stop being evil'Sep 14, 2010 at 20:24

A master class in understatement

With no fanfaronade they dropped a single sentence in a comment that should have been, not only an answer but, the answer. Not only the answer to this question, but the reference answer to an entire class of questions.

Breaking it down

What does strace -s9999 -f -o tmux.strace tmux -c true mean?

It's sort of like 2 commands in 1. The main command is tmux but it is given a "prefix"/"wrapper" called strace. This is very similar to running time df to measure how much time it takes to run the df command.

strace-s9999-f-o tmux.strace

$ man strace | grep -A1 ^NAMENAME       strace - trace system calls and signals$ man strace | grep -EA10 '^ *DESCRIPTION($| )'DESCRIPTION       In the simplest case strace runs the specified command until it  exits.   It       intercepts  and  records  the system calls which are called by a process and       the signals which are received by a process.  The name of each system  call,       its  arguments  and its return value are printed on standard error or to the       file specified with the -o option.       strace is a useful diagnostic, instructional, and  debugging  tool.   System       administrators,  diagnosticians and trouble-shooters will find it invaluable       for solving problems with programs for  which  the  source  is  not  readily       available  since  they  do not need to be recompiled in order to trace them.$ man strace | grep -EA4 '^ *-s($| )'       -s strsize       --string-limit=strsize                   Specify the maximum string size to print (the  default  is  32).                   Note  that  filenames  are not considered strings and are always                   printed in full.$ man strace | grep -EA6 '^ *-f($| )'       -f       --follow-forks                   Trace child processes as they are created  by  currently  traced                   processes as a result of the fork(2), vfork(2) and clone(2) sys‐                   tem calls.  Note that -p PID  -f  will  attach  all  threads  of                   process  PID  if  it  is  multi-threaded,  not  only thread with                   thread_id = PID.$ man strace | grep -EA8 '^ *-o($| )'       -o filename       --output=filename                   Write the trace output to  the  file  filename  rather  than  to                   stderr.   filename.pid  form  is used if -ff option is supplied.                   If the argument begins with '|' or '!', the rest of the argument                   is  treated as a command and all output is piped to it.  This is                   convenient for piping the debugging output to a program  without                   affecting  the redirections of executed programs.  The latter is                   not compatible with -ff option currently.

tmux-c true

$ man tmux | grep -EA4 '^ *-c($| [^ ]*$)'     -c shell-command                   Execute shell-command using the default shell.  If necessary,                   the tmux server will be started to retrieve the default-shell                   option.  This option is for compatibility with sh(1) when tmux                   is used as a login shell.$ man true | grep -A1 ^NAMENAME       true - do nothing, successfully

Why...

use -c with tmux?

Since we only really care about the startup process of tmux, we don't need to use it interactively. Therefore, we will instruct it to run a command in place of our normal interactive shell.

use true as the tmux command?

Because it exits very quickly and will keep our log short.

use -o tmux.strace with strace?

It's just file to store the output. Naming things is hard. It was not important to match the basename or the extension to anything else.

use -f with strace?

I'll assume you read the excerpt from the man page and not redundacate. Any reasonably complex application is going to create child processes. Since tmux is magnificently complex, we definitely want to strace those too.

use -s9999 with strace?

Since we are writing the output to a file, we don't need line truncated to fit our terminal width. Set it large. Deal with displaying it later if it's a problem.

Why do any of this?

If by any, you mean write this long-ass answer...

1. Because I train engineering teams and will point them to this answer.
2. Because Giles...
   • is freaking hero
   • had 0 upvotes on their comment for 10+ years
   • has gone unnoticed for too long, and I refuse to let it continue

If you don't see why this process is The Way to answer a question like"Any idea what's wrong with the loading of the conf?"then I would refer you to the XY Problem

It is not only questioners who fall into the XY Problem. It extremely common for answers to ~make this mistake~ do this also. In this case, many answers to this question instruct the reader how to load/reload a config file after the process has started. That suggests ignoring the fact the process fails to load the file as it should.

If you have made it this far, what I want you to take away is take it's only okay to ignore a failure if you are:

• moving past a blocker so that you can remove yourself as a blocker for others
• going to come back a fix the problem correctly later

The Take Away

Many problems can be distilled down to "What's wrong with the loading of the file?". That question should tickle a spidey-sense in the form of "Is it trying to load the file? What is happening when it tries?"

You can/should[?] use strace solve anything that can be be questioned as:

• What is this process trying to do?
• What is it experiencing when it tries to do that [thing it's trying to do]?

Often, it is not a failure of the process to do a thing, but a failure of the user expecting the process to do a thing. And, the process experiences a lot of things that the user is not informed of.

Enough with the abstract pontification. Let's consider an example:

$ tmux kill-server; rm tmux.strace; strace -s9999 -f -o tmux.strace tmux -c trueno server running on /tmp/tmux-1000/default$ ls -lh tmux*-rw-r--r-- 1 bruno bruno  21K Jun 11 10:58 tmux-client-18358.log-rw-r--r-- 1 bruno bruno 324M Jun 11 10:58 tmux-server-18360.log-rw-r--r-- 1 bruno bruno 342K Jun 11 13:48 tmux.strace$ grep 'tmux\.conf' tmux.strace521744 readlink("/etc/tmux.conf", 0x7ffe2c0446d0, 1023) = -1 ENOENT (No such file or directory)521744 readlink("/home/bruno/.tmux.conf", 0x7ffe2c0446d0, 1023) = -1 ENOENT (No such file or directory)521744 readlink("/home/bruno/.config/tmux/tmux.conf", 0x7ffe2c0446d0, 1023) = -1 ENOENT (No such file or directory)$ ls -lA /home/bruno/.config/tmux-rw-r--r-- 1 bruno bruno 307 Jun 10 20:29 tmux.config

Do you see it? Look at the output of the last 2 commands. Maybe it would help if I showed you how I fix it.

$ mv ~/.config/tmux/tmux.config \    ~/.config/tmux/tmux.conf$ tmux kill-server; rm tmux.strace; strace -s9999 -f -o tmux.strace tmux -c trueno server running on /tmp/tmux-1000/default$ grep -A2 '\.config/tmux/tmux\.conf' tmux.strace527872 openat(AT_FDCWD, "/home/bruno/.config/tmux/tmux.conf", O_RDONLY) = 8527872 newfstatat(8, "", {st_mode=S_IFREG|0644, st_size=307, ...}, AT_EMPTY_PATH) = 0527872 read(8, "set -sg escape-time 0\nset -g mode-keys vi\n\n# remap prefix from 'C-b' to 'C-f'\nset-option -g prefix C-f\nunbind-key C-b\nbind-key C-f send-prefix\n\n# split panes using | and -\nbind | split-window -h\nbind - split-window -v\n# bind \"\33OA\" # scroll natural up\n# bind \"\33OB\" # scroll natural down\nunbind '\"'\nunbind %\n\n", 4096) = 307

Note: That last line is the entire contents of my (previously misnamed) ~/.config/tmux/tmux.conf file.