Shell Scripting
- A few shell documentation gems
I'm hosting mirrors of these at my domain.
- Help creating custom fuzzy seach command script.
I want to interactively query nix pkgs using the nix-search command provided by
nix-search-cli
Not really experiaenced in cli tools any ideas to make this work ?
- New 2024 POSIX Standards specify features for sh and standard shell utilities
The document itself is paywalled like all the POSIX specifications, so here are some highlights courtesy of some comments at HackerNews, especially from a-french-anon.
- Configure your new computer or server with a convenience POSIX shell script, instead of doing it all manuallymaclong.deno.dev Configure Your New Machine with a Shell Script
Utilise POSIX Shell to take the hassle out of configuring your system each time you get a new computer or provision a server.
- CLI bookmarking utilitygithub.com GitHub - danhab99/cli-bookmarks: a cli utility for bookmarking directories
a cli utility for bookmarking directories. Contribute to danhab99/cli-bookmarks development by creating an account on GitHub.
- Is there a better algorithm for converting big binaries into decimal?
At the moment, I am stuck with using single-precision float, and double-precision float. So, the maximum represent-able value for single is 1<<24 while for double, it is 1<<53.
Because of this, I made the following script here - https://gist.github.com/Reptorian1125/71e3eec41e44e2e3d896a10f2a51448e .
Allow me to clarify on the script above. On the first part, rep_bin2dec does is to return the converted values into the status. So, when I do ${} or variable=${rep_bin2dec\ ???}, I get the status string.
On the second part, rep_bin2dec_base is the basis for getting rep_bin2dec to work. _rep_bin2dec_base prints the base_10M array into a string.
So, how does rep_bin2dec_base converts a big binary into big decimal?
-
If the binary image is less than dimension of 54, then the script will use 0b{} which allows me to directly convert binary to decimal, and 0b is a binary literal much in the same way that Python and C++ does it. From this point, it's pretty obvious on what to do there. So, if it less than dimension 54, this step 1 is pretty much done. If not, move on to step 2.
-
Convert the binary image as a image of base (1<<24) representing the value of that image. Note that there are two channels "[ output_value , y ]". y in this case represents the digit position in base (1<<24).
-
Make the converted image as a dynamic array image. This allows us to remove unused digits. You can look at step 2, and step 3 as converting a binary string into an array of base (1<<24) into a dynamic array. Also, note that start_value is stored. That's the very first digit.
-
Note that the number_of_decimals is the predicted number of characters after conversion of binary to decimal. And the, there's multi-threading that gets activated depending on the size of dynamic array image. decimal_conversion_array_size,result_conversion_array_size is used to define array size as they're temporary arrays to convert from base (1<<24) into base 10M. Finally, there's a new image which is going to be using base 10 million for easy printing, and set is used to add the first digit of base (1<<24) which will then be converted to base 10M.
-
On eval[-2], we are now processing the base (1<<24) image, and then convert it into base 10M. There's a implicit loop, so you can add a "for y" after begin(), and begin() can be seen as the setup code.
Some notes, copy() basically allows me to alter an array. In this case, opacity is negative, so it will add the multiplication of the positive opacity. If opacity was between 0-1, then it will get treated similar to how opacity of one layer alters a image. And the multiplication algorithm being used to convert between bases is Schönhage-Strassen multiplication, but without the FFT part.
So, here how that works.
9 9 x 1 9 _________ 81 81 9 9 _________ 1 8 8 1
Basically, it's long multiplication, and you can see that there's carrying of the remainder. 81 -> 1 (Remainder 8). 81 + 9 + R8 = 89 + 9 = 8 R ( 1+ 8 ) = 8 R 9. Then 9 + 9 is 18. So, you can see how this results in 1881.
- After the conversion to base 10M, depending on your inputs, it'll set the status value to the decimal representation or preserves it as a base 10M for easy printing with _rep_bin2dec_base after alteration.
There's some more details, but I find it really hard to explain this.
So, my question is what are some good algorithm to print out huge binaries as decimal? I know Python is insanely good at that, but I can't seem to understand how it does that so well. I know that they do involve conversion to base 2^30 or 1<<30.
At the moment, I can convert a 90000 digits binary in .35 s, and that's bad to what I seen in Python. It's really bad with 1M binary digits.
-
- Ksh Python Transpilergithub.com GitHub - dislux-hapfyl/pynksh: ksh python transpiler
ksh python transpiler. Contribute to dislux-hapfyl/pynksh development by creating an account on GitHub.
https://discord.gg/JT6RMMfP
- Script to automatically update hosts file from a filter listgist.github.com Script to automatically update hosts file from a filter list
Script to automatically update hosts file from a filter list - README.md
- Pulling variables from functions executed as conditions
I've written small bash scripts before, but I bit off a little more this time, and I'm trying to program a little terminal game.
Anyway, I've run into a weird behavior. I have a function that, among other things, writes values to an array and returns either 0 or 1.
If I write
echo "${name_of_array[@]}"
inside the function, I see the contents of the array printed to standard output.If I write
echo "${name_of_array[@]}"
outside the function immediately after executing the function, I see the contents of the array printed to standard output. (So, clearly the array is being treated as a global variable.)But if I write the following, regardless of which value the function returns, nothing is printed to standard output but an empty line.
if ! name_of_function ; then echo "${name_of_array[@]}" fi
Why is that the case? Doesn't name_of_function get executed when evaluating the if statement? Is this some special case where all variables become local?
I realize I could just assign the function's return value to a separate variable and then use that variable as the condition to the if statement, but it's less elegant, and it doesn't satisfy my curiosity. Is there any way to get the array out of the if statement alone?
It's also possible I'm an idiot and my problem is just some random punctuation somewhere.
- Using broot as an fzf-like path completer and interactive folder jumper in Zshandydecleyre.github.io Using broot as an fzf-like path completer and interactive folder jumper in Zsh
Broot is a great file manager TUI which can be adapted for very specific workflows.
Hello! I love broot. It's not my own project, but this blog post is, so feel free to send any questions or insults my way.
Previews:
- Nushell 0.90.1 | Nushell
Looks like v0.90 was skipped on nushell blog, so expect minor changes from these release notes
- Release fish 3.7.0 (released January 1, 2024)github.com Release fish 3.7.0 (released January 1, 2024) · fish-shell/fish-shell
This release of fish includes a number of improvements over fish 3.6.4, detailed below. Although work continues on the porting of fish internals to the Rust programming language, that work is not i...
This release of fish includes a number of improvements over fish 3.6.4, detailed below. Although work continues on the porting of fish internals to the Rust programming language, that work is not included in this release. fish 3.7.0 and any future releases in the 3.7 series remain C++ programs.
Notable improvements and fixes
- Improvements to the history pager, including:
- The history pager will now also attempt subsequence matches (#9476), so you can find a command line like
git log 3.6.1..Integration_3.7.0
by searching for gitInt. - Opening the history pager will now fill the search field with a search string if you’re already in a search (#10005). This makes it nicer to search something with
↑
and then later decide to switch to the full pager. - Closing the history pager with enter will now copy the search text to the commandline if there was no match, so you can continue editing the command you tried to find right away (#9934).
- Performance improvements for command completions and globbing, where supported by the operating system, especially on slow filesystems such as NFS (#9891, #9931, #10032, #10052).
- fish can now be configured to wait a specified amount of time for a multi-key sequence to be completed, instead of waiting indefinitely. For example, this makes binding
kj
to switching modes in vi mode possible. - The timeout can be set via the new
fish_sequence_key_delay_ms
variable (#7401), and may be set by default in future versions.
- Nushell v0.89 release
This release adds spreading of argument lists to command calls, better editor integration, and many bugfixes.
- Can I email or text myself through Python or bash?
cross-posted from: https://beehaw.org/post/10863052
> Noob question incoming, thanks in advance for any help with this! > > I have a specific use case in which I want to send an automated email or text to myself once a day (the message is different each time--otherwise I would just set an alarm, lol!). I'm running Pop_OS on an old desktop computer. Where I'm stuck is getting an email to successfully send from the command line. I'm looking for easy-to-follow instructions that would help me do that, and none of the articles or videos I've come across thus far have helped. > > I'm aware of Twilio and other services that send SMS messages, but I'm looking for something free. Especially since I only need to text one person (myself), and infrequently at that. > > Below is my attempt to send an email with the telnet command. Nothing ever came through... > >
> XXXXXXXX@pop-os:~$ telnet localhost smtp > Trying ::1... > Connected to localhost. > Escape character is '^]'. > 220 pop-os ESMTP Exim 4.95 Ubuntu Sun, 07 Jan 2024 15:12:28 -0500 > HELO gmail.com > 250 pop-os Hello localhost [::1] > mail from: XXXXXXXX@gmail.com > 250 OK > rcpt to: XXXXXXXX@gmail.com > 250 Accepted > data > 354 Enter message, ending with "." on a line by itself > Subject: Test > Body: Is this working? > . > 250 OK id=1rMZW4-0002dj-Uy > quit >
> - I made a Brainfuck interpreter within G'MIC (Shell-Like language for image processing)
Three things before I'll get to the relevant details.
- Brainfuck is a esoteric languages which uses 8 characters. I'll leave details here - https://en.wikipedia.org/wiki/Brainfuck
- G'MIC is a language largely inspired by bash languages and one other shell scripting language, and partly inspired by C++ for JIT compilation. It's two languages in one as in one outside of JIT and one inside of JIT. It's main purpose is image processing, and it can do 3D things too, basically image-related things. It's turing-complete, so making files has been done with it. Even making a executable compiled program is possible in practice (but, I would point to doing C++ and compile there instead).
- I am a G'MIC filters developer.
Anyways, I taken some time to code up a Brainfuck interpreter within G'MIC. It wasn't that hard to do once I understood what Brainfuck is as a language. I did one earlier than this, but I had to have users define inputs beforehand. Recently, I created rep_cin command to relieve users of doing that, and that is the closest to
input()
within Python orstd::cin
via C++.Anyways, here's the code to my Brainfuck interpreter: ``` #@cli run_brainfuck_it: brainfuck_file,'_enforce_numbers_input={ 0=false | 1=true },_size_of_array>0 #@cli : Interprets Brainfuck code file within G'MIC brainfuck_interpreter. #@cli : Default values: ,'_enforce_numbers_input=0','_size_of_array=512' run_brainfuck_it: skip ${2=0},${3=512} it $1 _brainfuck_interpreter $2,$3 um run_brainfuck_it,run_brainfuck,_brainfuck_interpreter,_brainfuck_interpreter_byte_input #@cli run_brainfuck: brainfuck_code,'_enforce_numbers_input={ 0=false | 1=true },_size_of_array>0 #@cli : Interprets Brainfuck code within G'MIC brainfuck_interpreter. #@cli : Default values: ,'_enforce_numbers_input=0','_size_of_array=512' run_brainfuck: skip ${2=0},${3=512} ('$1') _brainfuck_interpreter $2,$3 um run_brainfuck_it,run_brainfuck,_brainfuck_interpreter,_brainfuck_interpreter_byte_input _brainfuck_interpreter: # 1. Convert image into dynamic image resize 1,{whd#-1},1,1,-1 ({h}) append y # Convert string images into dynamic image name[-1] brainfuck_code # Name image into brainfuck_code
# 2. Remove unused characters eval " const brainfuck_code=$brainfuck_code; for(p=h#brainfuck_code-2,p>-1,--p, char=i[#brainfuck_code,p]; if(!(inrange(char,'+','.',1,1)||(find('<>[]',char,0,1)!=-1)), da_remove(#brainfuck_code,p); ); ); if(!da_size(#brainfuck_code), run('error inval_code'); ); da_freeze(#brainfuck_code); "
# 3. Evaluate brackets eval[brainfuck_code] >" begin(level=0;); i=='['?++level: i==']'?--level; if(level<0,run('error inv_bracks');); end(if(level,run('error inv_bracks');););"
1x2 # Create 2 images of 1x1x1x1. One image is for storing print out characters, and the other is to allow inputs. _arg_level=1
# 4. Create JIT code for executing brainfuck code. repeat h#$brainfuck_code { idx:=i[#0,$>]
if $idx==',' code_str.=run('$0_byte_input[-2]\ $1');ind_list[ind]=i#-2; continue fi if $idx=='.' code_str.=da_push(#-1,ind_list[ind]); continue fi if $idx=='+' code_str.=ind_list[ind]++;ind_list[ind]%=256; continue fi if $idx=='-' code_str.=ind_list[ind]--;ind_list[ind]%=256; continue fi if $idx=='<' code_str.=if(!inrange(--ind,0,$2,1,0),run("'error out_of_bound'");); continue fi if $idx=='>' code_str.=if(!inrange(++ind,0,$2,1,0),run("'error out_of_bound'");); continue fi if $idx=='[' code_str.=repeat(inf,if(!ind_list[ind],break();); continue fi if $idx==']' code_str.=); fi }
# 5. Execute created JIT code. v + and v - is used to change verbosity level, not part of JIT execution. e[] is used to print into console. v + eval >begin(ind=0;ind_list=vector$2(););$code_str;end(da_freeze(#-1);); v -
# 6. Print out executed code result v + e[$^] "Brainfuck Output: "{t} v - remove _brainfuck_interpreter_byte_input: repeat inf { wait # For some reason, I had to add this to make this code work!
if $> rep_cin "Brainfuck Interpreter - Wrong Input! Insert Integer for Argument#"$_arg_level": " else rep_cin "Brainfuck Interpreter - Enter Argument#"$_arg_level" (Integers Only): " fi
if $1 input:=(${}%208)+_'0' else input=${} fi
if isint($input) break fi }
if $1 v + e[$^] "Brainfuck Interpreter Inserted Argument#"$arg_level": "{$input-'0'} v - else input%=256 v + e[$^] "Brainfuck Interpreter Inserted Argument#"$_arg_level": "$input" :: "{
$input
} v - fi_arg_level+=1 f[-1] $input ```
And the CLI test:
C:\Users\User\Documents\G'MIC\Brainfuck Interpreter>gmic "brainfuck_interpreter.gmic" run_brainfuck \">,>,<<++++++[>-------->--------<<-]>[>[>+>+<<-]>[<+>-]<<-]>[-]>+>>++++++++++<[->-[>>>]++++++++++<<+[<<<]>>>>]<-<++++++++++>>>[-<<<->>>]<<<<++++++[>++++++++>[++++++++>]<[<]>-]>>[.<<]<[<<]>>.\",1 [gmic]./ Start G'MIC interpreter (v.3.3.3). [gmic]./ Input custom command file 'brainfuck_interpreter.gmic' (4 new, total: 4806). [gmic]./ Brainfuck Interpreter Inserted Argument#1: 31 [gmic]./ Brainfuck Interpreter Inserted Argument#2: 3 [gmic]./ Brainfuck Output: 93 [gmic]./ End G'MIC interpreter.
- Javascript code challengechat-to.dev Chat Rooms and Programming Content | Chat-to.dev
Join chat rooms and explore programming content on Chat-to.dev.
If you have the answer please leave it here in the comments #javascript #php #programming
- olets/zsh-test-runner: Straight-forward tests and coverage reports for Zsh (and emulated csh, ksh, sh)github.com GitHub - olets/zsh-test-runner: Straight-forward tests and coverage reports for zsh and —under zsh's emulation— csh, ksh, and sh
Straight-forward tests and coverage reports for zsh and —under zsh's emulation— csh, ksh, and sh - GitHub - olets/zsh-test-runner: Straight-forward tests and coverage reports for zsh and —under...
This is not my work, but the author (same author as zsh-abbr) posted it elsewhere and it looks good to me.
In his words:
> What is zsh-test-runner? A simple testing framework for zsh, and to a degree —thanks to zsh's emulation of other shells— csh, ksh, and sh.
> The immediately noticeable difference between zsh-test-runner and other shell script unit test frameworks is it doesn't have a DSL. zsh-test-runner relies entirely on the shell's own testing. For those familiar with other frameworks: nothing like ShellSpec's
Describe … When call … The output should
, or shUnit2'sassertEquals
, or ZUnit'sassert
; zsh-test-runner is closer to Bats if you were to restrict yourself to core and not use helper libraries (there's nothing like bats-assert'sassertEquals
or bats-file'sassert_dir_exists
).> Why no special syntax? It means there's little new to learn— For example, if you know how to test numeric equality in your shell, you know how to test equality in zsh-test-runner; if you don't, there are community resources available. It means every possible test is supported equally out of the box— zsh-test-runner is a newcomer, but there are no "shoot my assertion method isn't supported" blockers. It means the cost of porting homegrow framework-less tests to zsh-test-runner is about as low as can be— generally speaking,
my_cool_test_code
becomesztr test 'my_cool_test_code'
. It means tests can live comfortably in one LOC, making zsh-test-runner pleasant to use in the terminal. - After a multi-year hiatus, an org transfer with new admins and maintainers, jq 1.7 has been released!github.com Release jq 1.7 · jqlang/jq
After a five year hiatus we're back with a GitHub organization, with new admins and new maintainers who have brought a great deal of energy to make a long-awaited and long-needed new release. We'r...
Check out the newest version of everyone's favorite^[citation needed] command line json processing tool!
Highlights include significant speed improvements (>10x on some of my workloads), new flags, new builtins, and a litany of bugfixes.
- How do you shell expand your variables and why?
x-post from !programming@programming.dev: https://programming.dev/post/2165136
- .bin/check_installed.sh:fc:3: no such event: 1 when trying to run a set of commands from script that work fine when ran in the terminal directly
Hello everyone. I wrote this command in the terminal directly and got the desired and expected output - that being the last 50 occurrences of me installing or removing a package with pacman or yay:
history | grep -e 'pacman -S\s' -e 'pacman -R\s' -e 'yay -S\s' -e 'yay -R\s' | tail -n 50 > ~/history_installed
I now want to make this runnable as a script for obvious reasons, but when add it to a script and run it I get the following error:
/home/user/.bin/check_installed.sh:fc:3: no such event: 1
Here is my entire script: ``` #!/bin/zsh
{history | grep -e 'pacman -S\s' -e 'pacman -R\s' -e 'yay -S\s' -e 'yay -R\s' | tail -n 50} > ~/history_installed ```
Note: /home/user/.bin is in my path. Verified by successfully running another script in there from a different location.
Please help me figure this out if you could. I am running zsh with oh-my-zsh. Thanks in advance!
- announcement : link-batch.zsh, to generate symlinks batches
cross-posted from: https://lemmy.world/post/1838642
> link-batch is a minimalist script that generate symlinks from a list in a text file. Usage : > >
sh > link-batch.zsh link-list.txt >
> > wherelink-list.txt
contains two columns : the first one for the links and the second one for the targets. Example : > >> ~/.config/kitty ~/myfiles/config/kitty > ~/.config/nvim ~/myfiles/config/neovim > ~/.config/MuseScore ~/myfiles/config/MuseScore/$HOST > ... >
> > The two columns must be separated by a tab. > > Shell vars like$HOME
or$HOST
are evaluated to their values. > > Can be used to quickly deploy all home links in a fresh box. - Conway's Game of Life in POSIX Shellgithub.com GitHub - narshee/conway.sh: Conway's Game of Life in POSIX Shell
Conway's Game of Life in POSIX Shell. Contribute to narshee/conway.sh development by creating an account on GitHub.
This is working just fine, but it's not finished. However I am not interested in working on it anymore.
- What are the most inscrutable lines of $SHELL you've ever written?
I'm sure some of you have absolute monstrosities of sigils (I know I do, in my .zshrc alone). Post them without context, and try and guess what other users's lines are. If you want to provide context or guess, use the markdown editor to spoiler-tag your guesses and explanations!
- 2048 game I made in POSIX Shellgithub.com GitHub - narshee/2048.sh: 2048 in POSIX Shell
2048 in POSIX Shell. Contribute to narshee/2048.sh development by creating an account on GitHub.
- do you use file extensions for your scripts?
i'm pretty new to the shell scripting world and not sure, if i should give my scripts a .sh or .bash extension.
not sure what the pros and cons are.
- jq-shell: Functions for bringing json structures useful to shellgithub.com GitHub - xPMo/jq-shell: Functions for making json structures useful to shell
Functions for making json structures useful to shell - GitHub - xPMo/jq-shell: Functions for making json structures useful to shell
This came out of playing around with
curl
and the Lemmy API. I wanted a repeatable way to declare a bunch of shell parameters from a json object, so I wrote ajq
module! Take a look at the README for examples, or once you install, runshell source <( curl 'https://programming.dev/api/v3/community?id=267' | jq -r 'import "shell" as shell; .community_view | shell::param' )
Currently this project only provides a
param
function, and only supports Bash and Zsh. If I think of other useful helper functions, I may add them to the project.This uses
_
as a default prefix for each parameter. This is so things like{"PATH": "..."}
generatetypeset _PATH='...'
and don't mangle your $PATH, for example. The prefix can be customized by passing in a string:param("mypfx")
.Before declaring a
1.0
, I'm planning on expanding shell support, and have the function determine how to output using the the$SHELL
parameter:```fish
This does not work (yet!)
eval (curl $URL | jq -r ' import "shell" as shell; "fish" as $SHELL | .community_view | param ') ```
Always up for feedback!
- Fuzzy-switching sesssion in tmux scriptjdhao.github.io Fuzzy-switching Tmux Sessions with Ease
During my work, I use multiple Tmux sessions to manage different projects. When I create a new session, I usually give it unique name so that I know that the project is about via the session name.
I found this post really helpful to anyone who is using tmux
- [ysh] (2/3) Sketches of YSH Features
YSH, or the shell formally known as oil, is touted as a possible upgrade path from Bash.
This is the first in a three-part series of posts re-introducing the language.
- Reviewing YSH
- Sketches of YSH Features (you are here)
- YSH, Narrow Waists, and Perlis-Thompson Problems (Not yet released)
- [ysh] (1/3) Reviewing YSH
YSH, or the shell formally known as oil, is touted as a possible upgrade path from Bash.
This is the first in a three-part series of posts re-introducing the language.
- Reviewing YSH (you are here)
- Sketches of YSH Features
- YSH, Narrow Waists, and Perlis-Thompson Problems (Not yet released)
- [zsh] Builtin Option Parsing in Zshxpmo.gitlab.io Using Zparseopts
tl;dr: Always use the flags -D and -E (and -F if available). Add a colon to indicate an argument is required (similar to getopt)
- Tracing is your friend!
A huge part of programming in general is learning how to troubleshoot something, not just having someone else fix it for you. One of the basic ways to do that is by tracing. Not only can this help you figure out what your script is doing and how it's doing it, but in the event that you need help from another person, posting the output can be beneficial to the person attempting to help.
(If your shell isn't listed and you know how to enable tracing, comment below and I'll add it to the table!)
Shell | How to enable tracing ---|--- Bash |
set -x
orset -o xtrace
Fish |set fish_trace on
sh |set -x
Zsh |set -x
orsetopt xtrace
Also, writing scripts in an IDE that supports your language. syntax highlighting can immediately tell you that you're doing something wrong.
If an IDE isn't an option and you're using Bash or Sh, you can paste your script into https://www.shellcheck.net/
(Inspired by this post on /r/bash)
- Here's a sed tutorial I wrote a few years agoxpmo.gitlab.io Intermediate Sed Usage
If you’ve been using Linux for any small length of time, you’ve likely used sed before. Most of the time, you’ve seen it in the form of sed "s/find/replace/g", so you simply go to it whenever there’s a replacement you want to do. But sed stands for stream editor, and as a tool it can do more than ju...
Feedback is still welcome, either here or as Merge Requests to my Gitlab Pages repo.