For all UNIX-based systems

See this excellent resource.

Fixing “zsh: no matches found” error when using rsync or scp

If you’re getting errors like zsh: no matches found when executing commands like

rsync -avz server.example.com:* .

That is because you need to quote special characters like *:

rsync -avz server.example.com:"*" .

Fixing mysqldump errors when trying to export from the command line

When using mysqldump on the command line (for example when exporting your database for a backup) you might get this error:

mysqldump: Error: 'Access denied; you need (at least one of) the PROCESS privilege(s) for this operation' when trying to dump tablespaces

This can be fixed by running this MySQL command (replace rsyncer with whatever user you’re using)

GRANT PROCESS, SELECT, LOCK TABLES ON *.* TO 'rsyncer'@'localhost';

Kudos to this blog.

Export and import all databases in MySQL or MariaDB

Export all databases:

mysqldump -u root -p --all-databases > all_databases.sql

Import the resulting file:

mysql -u root -p < all_databases.sql

Use sudo without typing in your password everytime

Edit the sudoers (sudo visudo) then change the line for the sudoers groep to this:

%sudo   ALL=(ALL:ALL) NOPASSWD: ALL

Login with SSH to an unreachable server via another server

Host          internal.hostname.tld
User          user
HostName      internal.hostname.tld
ProxyCommand  ssh user@external.hostname.tld nc %h %p 2> /dev/null

Weird rsync errors

Errors like this could be anything:

rsync error: error in rsync protocol data stream

But make sure your SSH keys have proper permissions

chmod -R 700 ~/.ssh

Convert a batch of image files

Use mogrify to convert a bunch of JPG files to fit within a 1500×1500 frame (take longest side), and set quality to 60%

mogrify -resize 1500x1500 -quality 60 *.jpg

Same thing, but crop the images as well so they fit in the 1500×1500 frame, cropped from the center of the image.

mogrify -resize 1500x1500^ -crop 1500x1500+0+0 -gravity center *.jpg

Convert pages from a multi-page PDF to separate JPG files (3000px longest side), trimming excess whitespace

convert -resize 3000x doc.pdf -trim image-%d.jpg

Convert a set of image files to a PDF

convert *.png doc.pdf

Yes, it really is that simple.

Sequentially rename a bunch of files

Use the rename command to sequentially number a bunch of JPG files to 01.jpg, 02.jpg, etc.

rename -N 01 's/.*/$N.jpg/' *

Append both stdout and stderr to a file in Bash

command >> logfile.log 2>&1

Ffmpeg: video converting and editing

Basic use of ffmpeg

ffmpeg -i source.extension dest.extension

Resize to 600px wide

ffmpeg -i in.mov -vf scale=600:-1 out.mp4

Crop a video with landscape aspect-ratio to a square from the center

ffmpeg -i in.mov -filter:v "crop=in_h:in_h:((in_w/2)-(in_h/2)):0" out.mp4

Framerate to 24fps

ffmpeg -i in.mov -r 24 out.mp4

Bitrate to 2mb/s

ffmpeg -i in.mov -b:v 2000k out.mp4

Grab a frame from 1 second into the video and convert to jpg (note that .png also works)

ffmpeg -i in.mov -ss 00:00:01 -vframes 1 out.jpg

Make sure it works in Quicktime

ffmpeg -i in.mov -pix_fmt yuv420p out.mp4

Convert from a movie to a series of JPG’s (note that -q:v 2 is only needed for JPG)

ffmpeg -i clip.mp4 -q:v 2 "frames/%04d.jpg"

Grab a frame from every second of a clip and save to jpg

ffmpeg -i clip.mp4 -vf fps=1 frames/frame-%d.jpg

Same, but now every minute

ffmpeg -i clip.mp4 -vf fps=1/60 frames/frame-%d.jpg

Convert a bunch of images to a movie clip

ffmpeg -r 24.89 -f image2 -s 1280x720 -i "frames/%04d.jpg" -vcodec libx264 -crf 25 -pix_fmt yuv420p movie.mp4

Convert a MP3 to a MP4 with a single image

ffmpeg -loop 1 -y -i image.jpg -i music.mp3 -shortest -pix_fmt yuv420p output.mp4

Add MP3 audio to a movie and save as MP4 (given the audio is wav, i had less luck with mp3)

ffmpeg -i a.mp4 -i a.wav -c copy -map 0:v:0 -map 1:a:0 -shortest -c:a aac -b:a 192k b.mp4

Fix out of sync audio/video (note itsoffset parameter)

ffmpeg -i original.avi -itsoffset 0.2 -i original.avi -map 0:0 -map 1:1 -acodec copy -vcodec copy synced.avi

Download movies from YouTube and other video services

Use the brilliant youtube-dl.

youtube-dl -f mp4 https://www.youtube.com/watch?v=bMj3Pc5I078

Directly convert a movie at an URL to mp3

youtube-dl -x --audio-format mp3 https://www.youtube.com/watch?v=bMj3Pc5I078

Batch run a command on all files in a directory

for f in *.ext; do command $f; done

Search for a string appearing in files in directory and all subdirectoris, ignoring case

grep -lir some_string .

The same, but only in files with a certain extension

grep -lir --include \*.py some_string .

Get total size of directories, reporting only on the current level

du -h -d1

Query DNS records

Query the A record for example.com

dig example.com A

Query the MX record(s) for example.com

dig example.com MX

Get the result as just the IP instead of the whole query/answer stuff

dig example.com A +short

Linux-specific

Fix those stupid bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8) warnings

$ vim /etc/locale.gen

Uncomment the en_US.UTF-8 line

$ locale-gen

If you’re still getting errors, add these two lines to your bashrc or bash_profile on the machine you’re connecting from:

# Setting for the new UTF-8 terminal support in Lion
export LC_CTYPE=en_US.UTF-8
export LC_ALL=en_US.UTF-8

Add user to group

usermod -aG group user

Print out the full path of a file or directory

readlink -f file.txt

macOS-specific

Pipe outputs to the clipboard

ls | pbcopy

Find stuff on the command line using spotlight

mdfind