General Advice for shell scripts

What do you advice for shell usage?

  • Do you use bash? If not, which one do you use? zsh, fish? Why do you do it?
  • Do you write #!/bin/bash or #!/bin/sh? Do you write fish exclusive scripts?
  • Do you have two folders, one for proven commands and one for experimental?
  • Do you publish/ share those commands?
  • Do you sync the folder between your server and your workstation?
  • What should’ve people told you what to do/ use?
  • good practice?
  • general advice?
  • is it bad practice to create a handful of commands like podup and poddown that replace podman compose up -d and podman compose down or podlog as podman logs -f --tail 20 $1 or podenter for podman exec -it “$1” /bin/sh?

Background

I started bookmarking every somewhat useful website. Whenever I search for something for a second time, it’ll popup as the first search result. I often search for the same linux commands as well. When I moved to atomic Fedora, I had to search for rpm-ostree (POV: it was a horrible command for me, as a new user, to remember) or sudo ostree admin pin 0. Usually, I bookmark the website and can get back to it. One day, I started putting everything into a .bashrc file. Sooner rather than later I discovered that I could simply add ~/bin to my $PATH variable and put many useful scripts or commands into it.

For the most part I simply used bash. I knew that you could somehow extend it but I never did. Recently, I switched to fish because it has tab completion. It is awesome and I should’ve had completion years ago. This is a game changer for me.

I hated that bash would write the whole path and I was annoyed by it. I added PS1="$ " to my ~/.bashrc file. When I need to know the path, I simply type pwd. Recently, I found starship which has themes and adds another line just for the path. It colorizes the output and highlights whenever I’m in a toolbox/distrobox. It is awesome.

nawordar,
  • Fish. Much, much saner defaults.
  • I am writing #!/usr/bin/env sh for dead simple scripts, so they will be a tiny bit more portable and run a tiny bit faster. The lack of arrays causes too much pain in longer scripts. I would love to use Fish, but it lacks a strict mode.
  • No, why would I?
  • I used to share all my dotfiles, scripts included, but I was too afraid that I would publish some secrets someday, so I stopped doing that. For synchronizing commands, aliases and other stuff between computers I use Chezmoi.
  • To use Fish instead of fighting with start up time of Zsh with hundreds of plugins
  • Always use the so-called “strict mode” in Bash, that is, the set -euo pipefail line. It will make Bash error on non-zero exit code, undefined variables and non-zero exit codes in commands in pipe. Also, always use shellcheck. It’s extremely easy to make a mistake in Bash. If you want to check the single command exit code manually, just wrap it in set +e and set -e.
  • Consider writing your scripts in Python. Like Bash, it also has some warts, but is multiplatform and easy to read. I have a snippet which contains some boilerplate like a main function definition with ArgumentParser instantiated. Then at the end of the script the main function is called wrapped in try … except KeyboardInterrupt: exit(130) which should be a default behavior.
  • Absolutely not a bad practice. If you need to use them on a remote server and can’t remember what they stand for, you can always execute type some_command. Oh, and read about abbreviations in Fish. It always expands the abbreviation, so you see what you execute.
MigratingtoLemmy,

I use sh to attempt to keep it compatible with POSIX systems.

I use pain bash. Never really tried zsh and fish, since most of my Linux work is on servers and I don’t really care for extra features.

I try and write idempotent scripts when possible.

I wouldn’t create those aliases on a fleet because writing them to the configuration file of your shell in an idempotent fashion is hacky and my VMs are like cattle.

jbd,

I use fish shell only now. Used to only write bash, but I’ve started writing some fish scripts. I wouldn’t try to plan too much WRT shell scripting up front. Just fix your pain points as you go.

EpicVision,

Do you use bash? If not, which one do you use? zsh, fish? Why do you do it?

Mostly fish, because it just feels much more modern than bash, it has good built-in autocomplete and I don’t have to install millions of plugins like of zsh.

Do you write #!/bin/bash or #!/bin/sh? Do you write fish exclusive scripts?

#!/usr/bin/env bashOccasionally I also write fish scripts. Just replace sh with fish.

What should’ve people told you what to do/ use?

zoxide

general advice?

As @crispy_kilt already suggested, use shellcheck.

is it bad practice to create a handful of commands like podup and poddown that replace podman compose up -d and podman compose down or podlog as podman logs -f --tail 20 $1 or podenter for podman exec -it “$1” /bin/sh?

I don’t think so

crispy_kilt,

Use shellcheck

llii,

shellcheck

That looks useful.

www.shellcheck.net

Pantherina,

Yes fish is great. It has some special syntax for functions, I will add my configs soo.

set fish_greeting is useful to silence it.

User scripts can go to ~/.local/bin which is already in the path.

You can split up your shell configs into topics, and put them into ~/.config/fish/conf.d/abc.conf

redxef,
  • I usually use bash/python/perl if I can be sure that it will be available on all systems I intend to run the scripts. A notable exception for this would be alpine based containers, there it’s nearly exclusively #!/bin/sh.
  • Depending on the complexity I will either have a git repository for all random scripts I need and not test them, or a single repo per script with Integrationtests.
  • Depends, if they are specific to my setup, no, otherwise the git repository is public on my git server.
  • Usually no, because the servers are not always under my direct control, so the scripts that are on servers are specific to that server/the server fleet.
  • Regarding your last question in the list: You do you, I personally don’t, partly because of my previous point. A lot of servers are “cattle” provisioned and destroyed on a whim. I would have to sync those modifications to all machines to effectively use them, which is not always possible. So I also don’t do this on any personal devices, because I don’t want to build muscle memory that doesn’t apply everywhere.
ace_garp,
@ace_garp@lemmy.world avatar

Yes, using bash on all boxen.

Scripts start with #!/bin/sh ,because, that gives quicker execution times.

Any simple aliases, I put in .bash_aliases

Tried tcsh and zsh around 30yrs ago, all bash since then.

Hammerheart,

Do you have to chmod all your scripts when you include the shebang? Or do you have it configured to save with the right permissions?

ace_garp,
@ace_garp@lemmy.world avatar

I chmod 755 each manually. I’ve never tried the automatic way, sounds easier.

TCB13,
@TCB13@lemmy.world avatar

Do you use bash? Yes because it is everywhere and available by default.

wuphysics87,

Several things

  • write bash and nothing else (except posix sh)
  • find a good way to take notes. It shouldn’t be in your bashrc
  • only write fish for fish config
  • use $!/usr/bin/env bash
GravitySpoiled,

Good idea I added a “iwish” command a while ago. Whenever I am pissed about gnome not being able to do something, or anything else that didn’t work as it should, I wrote “iwish gnome had only one extension app” and it would add a new line to my wishlist.md Maybe it would be good for notes too. inote bla

Hammerheart,

I love thay idea im gonna implement it tonight

possiblylinux127,

You are way over thinking it.

bss03,

I primarily operate in strict standard compliance mode where I write against the shell specifications in the lastest Single Unix Specification and do not use a she-bang line since including one results in unspecified, implementation-defined behavior. Generally people seem to find this weird and annoying.

Sometimes I embrace using bash as a scripting language, and use one of the env-based she-bangs. In that case, I go whole-hog on bashisns. While I use zsh as my interactive shell, even I’m not mad enough to try to use it for scripts that need to run in more than one context (like other personal accounts/machines, even).

In ALL cases, use shellcheck and at least understand the diagnostics reported, even if you opt not to fix them. (I generally modify the script until I get a clean shellcheck run, but that can be quite involved… lists of files are pretty hard to deal with safely, actually.)

gnuhaut,

Btw, if you ever wondered why Debian uses dash as /bin/sh (the switch was a bit annoying at the time), I think the reasoning was something like this:

  • dash is a bit faster, which might have saved a second or two on boot times (this was before systemd). Same applies to compilation times, configure scripts run faster with dash.
  • A bunch of #!/bin/sh scripts in Debian did not actually work if you replaced /bin/sh with another shell, which I guess some people wanted to do. Making dash the default /bin/sh forced everyone to fix their scripts.

Also some history on the abomination that is m4sh, famously used by GNU autoconf configure.ac scripts. Apparently when autoconf was released in 1991, there were still some Unix systems that shipped some 70s shells as the default /bin/sh. These shells do not support shell functions, which makes creating any sort of shell programming library pretty much impossible (I guess you could make a folder full of scripts instead of functions). They decided to use m4 preprocessor macros instead, as a sort of poor man’s replacement for functions.

In hindsight, it wish they had told commercial Unix sysadmins to install a proper /bin/sh or gtfo. But the GNU people thought it was important to make it as easy as possible to install free software even on commercial Unices.

atzanteol,

Shell scripts are one of the things that makes Linux what it is. They’re relatively easy to create, powerful, etc. It was the thing that drove me to it from Windows in the first place.

One thing I would recommend against is creating dozens of utility scripts and/or aliases for things you run frequently. I have found it’s much better in the long-run to simply learn the “proper” commands and switches. If you use them often enough you start to type them very quickly. When you create helpers you start to learn your own ecosystem and will be lost on any system that doesn’t have your suite of helper apps installed.

There are exceptions to this to be sure (e.g. I always alias ‘l=ls -FhlA’) but I would specifically avoid the podup and poddown ones myself. I’ve gotten very quick at typing “docker run -it --rm foo” just by rote repetition.

You’re free to do as you like though. Maybe you’ll only run Linux on your own desktop so that’s all that matters. But something to keep in mind. I would at least learn the commands very well first and then later alias or script them for convenience.

draughtcyclist,

I agree. However… I do have a public repo with my helper scripts in case I need to set them up on a new machine. best of both worlds!

MigratingtoLemmy,

Welcome to the world of funny when you come across an airgapped server which doesn’t have the tools you use.

Eg: RHEL doesn’t have vim installed, now I can deal with nano but I’m way slower to do that. Luckily IaC has made my life somewhat easier

draughtcyclist,

Agreed, IaC has helped that process a lot. I just used to curse.

bionicjoey,

Do you use bash?

Personally I use Bash for scripting. It strikes the balance of being available on almost any system, while also being a bit more featureful than POSIX. For interactive use I bounce between bash and zsh depending on which machine I’m on.

Do you write #!/bin/bash or #!/bin/sh?

I start my shell scripts with #! /usr/bin/env bash. This is the best way of ensuring that the same bash interpreter is called that the user expects (even if more than one is present or if it is in an unusual location)

Do you have two folders, one for proven commands and one for experimental?

By commands, do you mean bash scripts? If so, I put the ones I have made relatively bulletproof in ~/bin/, as bash usually makes them automatically on the path with this particular folder name. If I’m working on a script and I don’t think it’s ready for that, or if it goes with a specific project/workflow, I will move it there.

Do you sync the folder between your server and your workstation?

No. I work on lots of servers, so for me it’s far more important to know the vanilla commands and tools rather than expect my home-made stuff to follow me everywhere.

good practice? general advice?

Pick a bash style guide and follow it. If a line is longer than 80 characters, find a better way of writing that logic. If your script file is longer than 200 lines, switch to a proper programming language like Python. Unless a variable is meant to interact with something outside of your script, don’t name it an all caps name.

is it bad practice to create a handful of commands like podup and poddown that replace podman compose up -d and podman compose down or podlog as podman logs -f --tail 20 $1 or podenter for podman exec -it “$1” /bin/sh?

This is a job for bash aliases.

aphlamingphoenix,

Good advice. I’ll add that any time you have to parse command line arguments with any real complexity you should probably be using Python or something. I’ve seen bash scripts where 200+ lines are dedicated to just reading parameters. It’s too much effort and too error prone.

bionicjoey,

It depends. Parsing commands can be done in a very lightweight way if you follow the bash philosophy of positional/readline programming rather than object oriented programming. Basically, think of each line of input (including the command line) as a list data structure of space-separated values, since that’s the underlying philosophy of all POSIX shells.

Bash is basically a text-oriented language rather than an object-oriented language. All data structures are actually strings. This is aligned with the UNIX philosophy of using textual byte streams as the standard interface between programs. You can do a surprising amount in pure bash once you appreciate and internalize this.

My preferred approach for CLI flag parsing is to use a case-esac switch block inside a while loop where each flag is a case, and then within the block for each case, you use the shift builtin to consume the args like a queue. Again, it works well enough if you want a little bit of CLI in your script, but if it grows too large you should probably migrate to a general purpose language.

bionicjoey, (edited )

Here’s a simple example of what I mean:


<span style="color:#323232;">#! /usr/bin/env bash
</span><span style="color:#323232;">
</span><span style="color:#323232;">while [[ -n $1 ]]; do
</span><span style="color:#323232;">  case $1 in
</span><span style="color:#323232;">    -a) echo "flag A is set" ;;
</span><span style="color:#323232;">    -b|--bee) echo "flag B is set" ;;
</span><span style="color:#323232;">    -c) shift; echo "flag C is $1" ;;
</span><span style="color:#323232;">    --dee=*) echo "flag D is ${1#--dee=}" ;;
</span><span style="color:#323232;">  esac
</span><span style="color:#323232;">  shift
</span><span style="color:#323232;">done
</span>

Showing how to do long flags with B and flags with parameters with C and D. The parameters will correctly work with quoted strings with spaces, so for example you could call this script with –dee=“foo bar” and it will work as expected.

MigratingtoLemmy,

Hoho, now do that in POSIX shell.

I had a rude awakening the day I tried it, but my scripts are bulletproof now (I think) so I don’t mind at this point

bionicjoey,

Imma be real, I never remember which parts of bash aren’t POSIX. Luckily it doesn’t matter in my line of work, but it’s good to be aware of if you have a job that often has you in machines running other types of UNIX.

MigratingtoLemmy,

Arguments don’t work the same way and POSIX doesn’t have the concept of arrays outside of @

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • linux@lemmy.ml
  • fightinggames
  • All magazines
    •