General Advice for shell scripts

What do you advice for shell usage?

  • Do you use bash? If not, which one do you use? zsh, fish? Why do you do it?
  • Do you write #!/bin/bash or #!/bin/sh? Do you write fish exclusive scripts?
  • Do you have two folders, one for proven commands and one for experimental?
  • Do you publish/ share those commands?
  • Do you sync the folder between your server and your workstation?
  • What should’ve people told you what to do/ use?
  • good practice?
  • general advice?
  • is it bad practice to create a handful of commands like podup and poddown that replace podman compose up -d and podman compose down or podlog as podman logs -f --tail 20 $1 or podenter for podman exec -it “$1” /bin/sh?

Background

I started bookmarking every somewhat useful website. Whenever I search for something for a second time, it’ll popup as the first search result. I often search for the same linux commands as well. When I moved to atomic Fedora, I had to search for rpm-ostree (POV: it was a horrible command for me, as a new user, to remember) or sudo ostree admin pin 0. Usually, I bookmark the website and can get back to it. One day, I started putting everything into a .bashrc file. Sooner rather than later I discovered that I could simply add ~/bin to my $PATH variable and put many useful scripts or commands into it.

For the most part I simply used bash. I knew that you could somehow extend it but I never did. Recently, I switched to fish because it has tab completion. It is awesome and I should’ve had completion years ago. This is a game changer for me.

I hated that bash would write the whole path and I was annoyed by it. I added PS1="$ " to my ~/.bashrc file. When I need to know the path, I simply type pwd. Recently, I found starship which has themes and adds another line just for the path. It colorizes the output and highlights whenever I’m in a toolbox/distrobox. It is awesome.

TCB13,
@TCB13@lemmy.world avatar

Do you use bash? Yes because it is everywhere and available by default.

jlsalvador, 

<span style="color:#323232;">#!/usr/bin/env bash
</span>

A folder dotfiles as git repository and a dotfiles/install that soft links all configurations into their places.

Two files, ~/.zshrc (without secrets, could be shared) and another for secrets (sourced by .zshrc if exist secrets).

clmbmb,

#!/usr/bin/env bash

This is the way!

GravitySpoiled,

why?

unlawfulbooger, (edited )

because bash isn’t always in /usr/bin/bash.

On macOS the version on /usr/bin/bash is very old (bash 3 I think?), so many users install a newer version with homebrew which ends up in PATH, which /usr/bin/env looks at.

Protip: I start every bash script with the following two lines:


<span style="font-style:italic;color:#969896;">#!/usr/bin/env bash
</span><span style="color:#62a35c;">set</span><span style="color:#323232;"> -euo pipefail
</span>

set -e makes the script exit if any command (that’s not part of things like if-statements) exits with a non-zero exit code

set -u makes the script exit when it tries to use undefined variables

set -o pipefail will make the exit code of the pipeline have the rightmost non-zero exit status of the pipeline, instead of always the rightmost command.

GravitySpoiled,

Nice, thx!

bizdelnick,

/bin/sh is always /bin/sh.

clmbmb,

#!/usr/bin/env will look in PATH for bash, and bash is not always in /bin, particularly on non-Linux systems. For example, on OpenBSD it’s in /usr/local/bin, as it’s an optional package.

If you are sure bash is in /bin and this won’t change, there’s no harm in putting it directly in your shebang.

GravitySpoiled,

dotfiles

Thanks! I’ll check them out. I knew the cooncept existed but so far I didn’t dig deep into managing them. This is my start I guess wiki.archlinux.org/title/Dotfiles

mryessir,

Instead of a install skript, check out GNU stow. It does exactly that and you can interqctively choose which things to install/symlink.

possiblylinux127,

You are way over thinking it.

bss03,

I primarily operate in strict standard compliance mode where I write against the shell specifications in the lastest Single Unix Specification and do not use a she-bang line since including one results in unspecified, implementation-defined behavior. Generally people seem to find this weird and annoying.

Sometimes I embrace using bash as a scripting language, and use one of the env-based she-bangs. In that case, I go whole-hog on bashisns. While I use zsh as my interactive shell, even I’m not mad enough to try to use it for scripts that need to run in more than one context (like other personal accounts/machines, even).

In ALL cases, use shellcheck and at least understand the diagnostics reported, even if you opt not to fix them. (I generally modify the script until I get a clean shellcheck run, but that can be quite involved… lists of files are pretty hard to deal with safely, actually.)

gnuhaut,

Btw, if you ever wondered why Debian uses dash as /bin/sh (the switch was a bit annoying at the time), I think the reasoning was something like this:

  • dash is a bit faster, which might have saved a second or two on boot times (this was before systemd). Same applies to compilation times, configure scripts run faster with dash.
  • A bunch of #!/bin/sh scripts in Debian did not actually work if you replaced /bin/sh with another shell, which I guess some people wanted to do. Making dash the default /bin/sh forced everyone to fix their scripts.

Also some history on the abomination that is m4sh, famously used by GNU autoconf configure.ac scripts. Apparently when autoconf was released in 1991, there were still some Unix systems that shipped some 70s shells as the default /bin/sh. These shells do not support shell functions, which makes creating any sort of shell programming library pretty much impossible (I guess you could make a folder full of scripts instead of functions). They decided to use m4 preprocessor macros instead, as a sort of poor man’s replacement for functions.

In hindsight, it wish they had told commercial Unix sysadmins to install a proper /bin/sh or gtfo. But the GNU people thought it was important to make it as easy as possible to install free software even on commercial Unices.

gnuhaut, (edited )

I use bash as my interactive shell. When ~20 years ago or so I encountered “smart” tab completion for the first time, I immediately disabled that and went back to dumb completion, because it caused multi-second freezes when it needed to load stuff from disk. I also saw it refuse to complete filenames because they had the wrong suffix. Maybe I should try to enable that again, see if it works any better now. It probably does go faster now with the SSDs.

I tried OpenBSD at some point, and it came with some version of ksh. Seems about equivalent to bash, but I had to modify some of my .bashrc so it would work on ksh. I would just stick to the default shell, whatever it is, it’s fine.

I try to stick to POSIX shell for scripts. I find that I don’t need bashisms very often, and I’ve used systems without bash on them. Most bash-only syntax has an equivalent that will work on POSIX sh. I do use bash if I really need some bash feature (I recently wanted to set -o pipefail, which dash cannot do apparently, and the workaround is really annoying).

Do not use #!/bin/sh if you’re writing bash-only scripts. This will break on Debian, Ubuntu, BSD, busybox etc. because /bin/sh is not bash on those systems.

SpaceCadet, (edited )
@SpaceCadet@feddit.nl avatar

Do not use #!/bin/sh if you’re not writing bash-only scripts

Actually #!/bin/sh is for bourne shell compatible scripts. Bash is a superset of the bourne shell, so anything that works in bourne should work in bash as well as in other bourne compatible shells, but not vice versa. Bash specific syntax is often referred to as a “bashism”, because it’s not compatible with other shells. So you should not use bashisms in scripts that start with #!/bin/sh.

The trouble is that it is very common for distros to links /bin/sh to /bin/bash, and it used to be that bash being called as /bin/sh would change its behavior so that bashisms would not work, but this doesn’t appear to be the case anymore. The result is that people often write what they think are bourne shell scripts but they unintentionally sneak in bashisms… and then when those supposed “bourne shell” scripts get run on a non-bash bourne compatible shell, they fail.

gnuhaut,

Oh I wanted to say, “Do not use #!/bin/sh if you’re not writing bash-only scripts”. I think I reformulated that sentence and forgot to remove the not. Sorry about the confusion. You’re exactly right of course. I have run into scripts that don’t work on Debian, because the author used bashisms but still specified /bin/sh as the interpreter.

SpaceCadet,
@SpaceCadet@feddit.nl avatar

Oh I wanted to say, “Do not use #!/bin/sh if you’re not writing bash-only scripts”

Hah, I was wondering if that was wat you actually meant. The double negation made my head spin a bit.

I have run into scripts that don’t work on Debian, because the author used bashisms but still specified /bin/sh as the interpreter.

The weird thing is that man bash still says:


<span style="color:#323232;">When invoked as sh, bash enters posix mode after the startup files are read.
</span><span style="color:#323232;">...
</span><span style="color:#323232;">--posix
</span><span style="color:#323232;">    Change  the  behavior  of bash where the default operation differs from the POSIX standard to 
</span><span style="color:#323232;">    match the standard (posix mode). See SEE ALSO below for a reference to a document that details 
</span><span style="color:#323232;">    how posix mode affects bash's behavior.
</span>

But if you create a file with a few well known bashisms, and a #!/bin/sh shebang, it runs the bashisms just fine.

starman,
@starman@programming.dev avatar

That’s the way I do it:


<span style="color:#323232;">#!/usr/bin/env nix
</span><span style="color:#323232;">#! nix shell nixpkgs#nushell <optionally more dependencies>  --command nu
</span><span style="color:#323232;">
</span><span style="color:#323232;"><script content>
</span>

But those scripts are only used by me

thedeadwalking4242,

This is the way

daddyjones,
@daddyjones@lemmy.world avatar

Am I missing something - doesn’t bash have tab completion or of the box?

zwekihoyy,

hardly

bionicjoey,

It does. It’s not quite as fancy as the completion in fish/zsh which employ a TUI, but it’s reliable in most situations

danielquinn, (edited )
@danielquinn@lemmy.ca avatar

I recommend writing everything in Bourne shell (/bin/sh) for a few reasons:

  • Bash is more capable, which is nice, but if you’re fiddling with complex data structures, you probably should be using a more maintainable language like Python.
  • Bash is in most places, but crucially not everywhere. Docker-based deployments for example often use Ash which is very similar to Bash, but lacks support for arrays and a few other things.
  • Bourne’s limitations force you to rethink your choices regularly. If you find yourself hacking around a lack of associative arrays for example, it’s probably time to switch to a proper language.

Also two bits of advice.

  1. Use shellcheck. There’s a website that’ll check your script for you as well as a bunch of editor extensions that’ll do it in real time. You will absolutely write better, safer code with it.
  2. If your script exceeds 300 lines. Stop and rewrite it in a proper language. Your future self will thank you.
Quackdoc,
@Quackdoc@lemmy.world avatar

I make my scripts with modern shells in mind since they let you use arrays for arguments which is vastly superior to doing `` for continuing the script line, I give the example here docs.blissos.org/…/advanced-qemu-config/…

Dirk,
@Dirk@lemmy.ml avatar
  • Do you use bash? If not, which one do you use? zsh, fish? Why do you do it?
  • Do you write #!/bin/bash or #!/bin/sh? Do you write fish exclusive scripts?

I use bash, and I use #!/bin/bash for my scripts. Some are POSIX compliant, some have bashisms. But I really don’t care about bashisms, since I explicitly set the bash as interpreter. So no, no fish exclusive scripts, but some “bash exclusive” scripts. Since fish is aimed towards being used as interactive shell I don’t see a real reason to use it as interpreter for scripts anyways.

  • Do you have two folders, one for proven commands and one for experimental?
  • Do you publish/ share those commands?
  • Do you sync the folder between your server and your workstation?

I have my scripts in $HOME/.scripts and softlink them from a directory in $PATH. Some of the scripts are versioned using Git, but the repository is private and I do not plan sharing them because the repoand the scripts scripts contain some not-tho-share information and mostly are simply not useful outside my carefully crafted and specific environment. If I want to share a script, I do it individually or make a proper public Git repository for it.

Since my server(s) and my workstations have different use cases I do not share any configuration between them. I share some configuration between different workstations, though. My dotfiles repository is mainly there for me to keep track of changes in my dotfiles.

is it bad practice to create a handful of commands

It becomes bad practice if it is against your personal or corporate guidelines regarding best practices. While it is not particularly bad or insecure, etc. to create bash scripts containing a single command, maybe use an alias instead. The $1 is automatically the first parameter after typing the alias in the shell.


<span style="color:#62a35c;">alias </span><span style="font-weight:bold;color:#795da3;">podup</span><span style="font-weight:bold;color:#a71d5d;">=</span><span style="color:#183691;">"podman compose up -d"
</span><span style="color:#62a35c;">alias </span><span style="font-weight:bold;color:#795da3;">poddown</span><span style="font-weight:bold;color:#a71d5d;">=</span><span style="color:#183691;">"podman compose down"
</span><span style="color:#62a35c;">alias </span><span style="font-weight:bold;color:#795da3;">podlog</span><span style="font-weight:bold;color:#a71d5d;">=</span><span style="color:#183691;">"podman logs -f --tail 20"
</span>

Not quite sure about the podman syntax, if podman exec /bin/sh -it “$1” also works, you can use alias podenter="podman exec /bin/sh -it, Otherwise a simple function would do the trick.

exu, (edited )

I use Bash for scripts, though my interactive shell is Fish.

Usually I use #!/usr/bin/env bash as shebang. This has the advantage of searching your PATH for Bash instead of hardcoding it.

My folders are only differentiated by those in my PATH and those not.

Most of my scripts can be found here. They are purely desktop use, no syncing to any servers. Most would be useless there.

For good practice, I’d recommend using set -euo pipefail to make Bash slightly less insane and use shellcheck to check for issues.
This is personal preference, but you could avoid Bashisms like [[ and stick to POSIX sh. (Use #!/usr/bin/env sh then.)

With shortened commands the risk is that you might forget how the full command works. How reliant you want to be on those commands being present is up to you. I wouldn’t implement them as scripts though, just simple aliases instead.
Scripts only make sense if you want to do something slightly more complex over multiple lines for readability.

GravitySpoiled,

#/usr/bin/env bash typo? #!/usr/bin/env bash

thx for the tips!

I prefer single files over aliases since I can more easily manage each command.

exu,

You’re right, it’s #!

Strit,
@Strit@lemmy.linuxuserspace.show avatar

I use bash and I usually put /bin/bash in my scrtipts, because that’s where I know it works. /bin/sh is only if it works on many/all shells.

I don’t have many such scripts, so I just have one. I don’t really share them, as they are made for my usecase. If I do create something that I think will help others, then yes, I share them in git somewhere.

I do have a scripts folder in my Nextcloud that I sync around with useful scripts.

Some of your examples can probably just be made into aliases with alias alias_name=“command_to_run”.

GravitySpoiled,

thx! Why do you think that aliases are better for it?

I moved away from aliases because I have a neat command management where each command is one script.

Strit,
@Strit@lemmy.linuxuserspace.show avatar

I can’t speak for anyone else, but for me, it’s just one file to backup to keep all your custom commands (.bashrc) while it would be many files if you have a script for each.

I can’t see the benefit of having a script for just one command (with arguments) unless those arguments contain variables.

jherazob,
@jherazob@beehaw.org avatar

A good idea i have been spreading around relevant people lately is to use ShellCheck as you code in Bash, integrate it in your workflow, editor or IDE as relevant to you (there’s a commandline tool as well as being available for editors in various forms), and pass your scripts through it, trying to get the warnings to go away. That should fix many obvious errors and clean up your code a bit.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • linux@lemmy.ml
  • fightinggames
  • All magazines
    •