For all UNIX-based systems
See this excellent resource.
Fixing “zsh: no matches found” error when using rsync or scp
If you’re getting errors like zsh: no matches found
when executing commands like
rsync -avz server.example.com:* .
That is because you need to quote special characters like *:
rsync -avz server.example.com:"*" .
Fixing mysqldump errors when trying to export from the command line
When using mysqldump
on the command line (for example when exporting your database for a backup) you might get this error:
mysqldump: Error: 'Access denied; you need (at least one of) the PROCESS privilege(s) for this operation' when trying to dump tablespaces
This can be fixed by running this MySQL command (replace rsyncer
with whatever user you’re using)
GRANT PROCESS, SELECT, LOCK TABLES ON *.* TO 'rsyncer'@'localhost';
Kudos to this blog.
Export and import all databases in MySQL or MariaDB
Export all databases:
mysqldump -u root -p --all-databases > all_databases.sql
Import the resulting file:
mysql -u root -p < all_databases.sql
Use sudo without typing in your password everytime
Edit the sudoers (sudo visudo
) then change the line for the sudoers groep to this:
%sudo ALL=(ALL:ALL) NOPASSWD: ALL
Login with SSH to an unreachable server via another server
Host internal.hostname.tld
User user
HostName internal.hostname.tld
ProxyCommand ssh user@external.hostname.tld nc %h %p 2> /dev/null
Weird rsync errors
Errors like this could be anything:
rsync error: error in rsync protocol data stream
But make sure your SSH keys have proper permissions
chmod -R 700 ~/.ssh
Convert a batch of image files
Use mogrify
to convert a bunch of JPG files to fit within a 1500×1500 frame (take longest side), and set quality to 60%
mogrify -resize 1500x1500 -quality 60 *.jpg
Same thing, but crop the images as well so they fit in the 1500×1500 frame, cropped from the center of the image.
mogrify -resize 1500x1500^ -crop 1500x1500+0+0 -gravity center *.jpg
Convert pages from a multi-page PDF to separate JPG files (3000px longest side), trimming excess whitespace
convert -resize 3000x doc.pdf -trim image-%d.jpg
Convert a set of image files to a PDF
convert *.png doc.pdf
Yes, it really is that simple.
Sequentially rename a bunch of files
Use the rename command to sequentially number a bunch of JPG files to 01.jpg, 02.jpg, etc.
rename -N 01 's/.*/$N.jpg/' *
Append both stdout and stderr to a file in Bash
command >> logfile.log 2>&1
Ffmpeg: video converting and editing
Basic use of ffmpeg
ffmpeg -i source.extension dest.extension
Resize to 600px wide
ffmpeg -i in.mov -vf scale=600:-1 out.mp4
Crop a video with landscape aspect-ratio to a square from the center
ffmpeg -i in.mov -filter:v "crop=in_h:in_h:((in_w/2)-(in_h/2)):0" out.mp4
Framerate to 24fps
ffmpeg -i in.mov -r 24 out.mp4
Bitrate to 2mb/s
ffmpeg -i in.mov -b:v 2000k out.mp4
Grab a frame from 1 second into the video and convert to jpg (note that .png also works)
ffmpeg -i in.mov -ss 00:00:01 -vframes 1 out.jpg
Make sure it works in Quicktime
ffmpeg -i in.mov -pix_fmt yuv420p out.mp4
Convert from a movie to a series of JPG’s (note that -q:v 2 is only needed for JPG)
ffmpeg -i clip.mp4 -q:v 2 "frames/%04d.jpg"
Grab a frame from every second of a clip and save to jpg
ffmpeg -i clip.mp4 -vf fps=1 frames/frame-%d.jpg
Same, but now every minute
ffmpeg -i clip.mp4 -vf fps=1/60 frames/frame-%d.jpg
Convert a bunch of images to a movie clip
ffmpeg -r 24.89 -f image2 -s 1280x720 -i "frames/%04d.jpg" -vcodec libx264 -crf 25 -pix_fmt yuv420p movie.mp4
Convert a MP3 to a MP4 with a single image
ffmpeg -loop 1 -y -i image.jpg -i music.mp3 -shortest -pix_fmt yuv420p output.mp4
Add MP3 audio to a movie and save as MP4 (given the audio is wav, i had less luck with mp3)
ffmpeg -i a.mp4 -i a.wav -c copy -map 0:v:0 -map 1:a:0 -shortest -c:a aac -b:a 192k b.mp4
Fix out of sync audio/video (note itsoffset
parameter)
ffmpeg -i original.avi -itsoffset 0.2 -i original.avi -map 0:0 -map 1:1 -acodec copy -vcodec copy synced.avi
Speed up video
ffmpeg -i input.mkv -filter:v "setpts=0.5*PTS" output.mkv
Combine video filters
Just separate by comma
ffmpeg -i input.mp4 -r 24 -filter:v "setpts=0.75*PTS,scale=210:-1" output.mp4
Download movies from YouTube and other video services
Use the brilliant youtube-dl.
youtube-dl -f mp4 https://www.youtube.com/watch?v=bMj3Pc5I078
Directly convert a movie at an URL to mp3
youtube-dl -x --audio-format mp3 https://www.youtube.com/watch?v=bMj3Pc5I078
Batch run a command on all files in a directory
for f in *.ext; do command $f; done
Search for a string appearing in files in directory and all subdirectoris, ignoring case
grep -lir some_string .
The same, but only in files with a certain extension
grep -lir --include \*.py some_string .
Get total size of directories, reporting only on the current level
du -h -d1
Query DNS records
Query the A record for example.com
dig example.com A
Query the MX record(s) for example.com
dig example.com MX
Get the result as just the IP instead of the whole query/answer stuff
dig example.com A +short
Work with tmux
Start a session
tmux
Re-attach to a session (if there’s only one session running)
tmux a
Detach from a session
CTRL-B d
Page through a session
CTRL-B [
Use arrow keys
Linux-specific
Fix those stupid bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8) warnings
$ vim /etc/locale.gen
Uncomment the en_US.UTF-8 line
$ locale-gen
If you’re still getting errors, add these two lines to your bashrc or bash_profile on the machine you’re connecting from:
# Setting for the new UTF-8 terminal support in Lion
export LC_CTYPE=en_US.UTF-8
export LC_ALL=en_US.UTF-8
Add user to group
usermod -aG group user
Print out the full path of a file or directory
readlink -f file.txt
macOS-specific
Pipe outputs to the clipboard
ls | pbcopy
Find stuff on the command line using spotlight
mdfind