Category Archives: Computer

ZFS: automatic snapshots

ZFS: Setting Up Automatic Snapshots

I considered writing a script to take care of this. Luckily, before I got too far down that rabbit hole, I stumbled upon Bryan Drewery’s zfstools on GitHub.

The zfstools installs a few Ruby (programming language) scripts that make managing automatic ZFS snapshots really easy. While you can customize the snapshot retention, the defaults will probably serve you well.

The default snapshot schedule is to snapshot your selected ZFS datasets:

  • every 15 minutes – keeping the last 4 quarter-hour snapshots
  • every hour – keeping the last 24 hourly snapshots
  • every day – keeping 7 daily snapshots
  • every week – keeping 4 weekly snapshots
  • every month – keeping 12 monthly snapshots

The setup is easy on FreeBSD. It only takes 3 steps:

  1. install the scripts
  2. turn on auto snapshotting
  3. update the /etc/crontab file

Start by installing zfstools from the FreeBSD package system:

$ sudo pkg install zfstools

Then, determine which ZFS datasets to automatically be snapshot. Turn on auto-snapshotting for these ZFS datasets:

$ sudo zfs set com.sun:auto-snapshot=true ZPOOL/DATASET

Where ZPOOL is the name of the zpool and where DATASET is the name of the ZFS dataset that you want to be automatically snapshot.

Next, it is time to update your machine’s /etc/crontab file to run the script. If you have never touched your /etc/crontab file before, take a moment to read the manual page for this magical file:

$ man 5 crontab

Next, copy the 5 crontab lines from https://github.com/bdrewery/zfstools README.md file. Look for the lines that have tons of asterisks – the manual page listed above explains what these asterisks and the rest of the fields represent.

Also, make sure the /etc/crontab PATH line contains the directories where ruby and zfs-auto-snapshot live. On my machine these were:

  • /usr/local/bin/
  • /usr/local/sbin/

You can figure out what yours are by typing:

$ which ruby

and

$ which zfs-auto-snapshot

The PATH line in my /etc/crontab file ended up looking like this:

PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin

Wait for the clock to pass the quarter hour then look for the automatic snapshots:

$ zfs list -t snapshot

You will see automatic snapshots start to pile up.

Coursera: Programming for Everybody (Python)

Certificate of Completion - Programming for Everybody

Certificate of Completion – Programming for Everybody

I managed to get a lot done this summer. One of my two major projects (I’ll save the 2nd one for its own post) was to take a computer programming course. I took the University of Michigan’s Programming for Everybody (Python) course.

This 10-week course is designed to teach the basics of programming computers. I took this course through Coursera, the site that is the home to many massive open online courses.

Here’s what I liked about it:

  • professor wrote the book we used
  • got to learn all about foundation ideas of programming (many of which I learned a long time ago then promptly forgot)
  • some of the homework problems challenged my brain
  • cool educational technology combining slides, video, drawing, and screencasting helped in making the programming easier to understand
  • nice use of humor
  • all course materials were licensed through Creative Commons (CC-BY)
  • textbook was free
  • cool online auto-grader for coding homework assignments
  • aimed at beginner programmers
  • lecture videos were available at Coursera and within the iBooks version of the text

Now that I have taken this course, I plan to take another Python programming class. I am eyeing the course offered by MITx, Introduction to Computer Science and Programming Using Python, 6.00.1x. With work and my other college classes I don’t know if I’ll have enough time to take it this fall, but I hope to take it in the spring or next summer.

As an educational technology nerd, I really enjoyed taking this class. It was cool to see how the instructor used the different edtech tools to deliver, grade, and collaborate on this class.

I give Professor Charles Severance and his class an A.

Read about the Creative Commons Attribution license by clicking on the CC-BY image below:
Creative Commons Attribution License

bash tab completion in tmux on Ubuntu GNU/Linux

I installed tmux on my Ubuntu laptop. I was confused about why tab completion was not working.

Silly me, it turns out that the default shell on tmux is /bin/sh and not the typical Bash (/bin/bash) shell typically used in GNU/Linux.

The simple fix was adding this to my $HOME/.tmux.conf file to ensure subsequent tmux sessions used /bin/bash.

set-option -g default-shell /bin/bash

ISO/RTO Tech Demo Links

Edited on July 26, 2015 to specify which items I did not cover in the presentation.

I presented a short talk on educational technology at the annual ISO/RTO training working group meeting.
Here are links to the sites that I discussed:

* not covered in the presentation due to time constraints

One topic that I didn’t get to that deserves way more attention is Markdown. It is my plain text holy grail. Type one place, then use tools to magically convert from txt to pdf, html, docx, presentation, LaTeX, etc.
Here are some links about it.

1Password on Ubuntu Linux 14.10

Edited on July 26, 2015 to include instructions on enabling browser extensions (Thanks, Jonathan!)

If you are as deeply entrenched in the Apple ecosystem as I am you probably survive password hell by using 1Password.

What if you want to play around with Ubuntu Linux? There is no version of 1Password for Linux. There is one neat work-around that I will show you here: you can use Wine, a Windows emulator, to run 1Password on Ubuntu Linux. This will probably work on other *nix systems too.

Watch it!

Check out my screencast on YouTube! to see the installation.

Install notes

Assumptions

This guide assumes the user has:

  • Ubuntu Linux 14.10 64-bit version is installed on a computer or virutal machine
  • beginner-level familiarity of the Linux command line
  • an existing 1Password installation that is being synchronized via Dropbox (not iCloud)
  • 1Password version 4 for Windows is being installed

Overview

  • install Wine
  • Install & configure Dropbox
  • Download 1Password
  • Install 1Password in Wine
  • (NEW) Install browser extensions
  • Run 1Password (NEW) with browser extensions enabled

Install WINE

Open up a Terminal window and type in the following:

sudo apt-get install wine

The Ubuntu package installer will then figure out what dependencies Wine has and it will figure out what needs to be downloaded. It will present you with a summary of what is to be installed and then it will as you if you want to continue. Type in y to continue.

It will then present you with an end user license agreement (EULA) for using Microsoft fonts. Click ok to accept it. Note: you will need to use the Tab key to focus on the <Ok> on this and subsequent dialog screens.

Install and Configure Dropbox

Open the Ubuntu Software Center and search for “Dropbox”.

When it shows up, click on Dropbox, then click on the Install button and enter your password when prompted.

The Ubuntu Software Center will install Dropbox for you. When it is done, close the Software Center window.

Click on the new Dropbox icon in the Launcher. A Dropbox setup window will then open. Enter your account information. At this point you accept the default of syncing everything in your Dropbox account. Alternatively, you can click on the Advanced Settings link to choose specific folders to sync.

Download 1Password

Running 1Password on Ubuntu requires the Windows version. You can grab it at the AgileBits Downloads page.

Click on the Download It Now button.

In the resultant pop-up dialog box, click on the Save File button.

Depending on your browser, you will either get to choose the target download directory or it will go to the default directory.

Install 1Password in WINE

Go to wherever the download is stored now. (In my case, the 1Password installer went to my /home/eduardo/Downloads directory)

cd ~/Downloads

List the contents of the directory and look for the 1Password executable installer. It will be named something like 1Password-4.1.0.530.exe . Your filename will be different than this.

Make sure you use your 1Password filename in the following command.

Use Wine to install 1Password by entering this command:

wine 1Password-4.1.0.530.exe

This will install 1Password in your Wine directory and start the 1Password Windows installer.

Select the default install location. It will say it is installing it to C:\Program Files (x86)\1Password 4 but, in reality, it is installing it to the Wine folder in your home directory.

Next you are given the opportunity to create a Start Menu shortcut. Since you are on Ubuntu, there is not really a start menu. Check the checkbox for Don’t create a Start Menu folder.

Check the checkbox for Create a desktop icon if you want one and click the Next button.

Click on the Install button to begin the installation. Once it has been installed click on the Finish button to exit the installer and get back to the command line.

Run 1Password

The simplest way to run 1Password is to use the Desktop icon. If you chose to install a Desktop icon keep in mind that it will probably not show up until after a reboot of the system.

If you didn’t install a Desktop icon, you can start Dropbox in the command line.

NEW: Install browser extensions by navigating to the AgileBits browser extensions site:
https://agilebits.com/onepassword/extensions

Navigate to your Wine folder. Assuming you chose the default directory when installing 1Password, you should type the following into the command line:

cd ~/.wine/drive_c/Program\ Files\ \(x86\)/1Password\ 4/

Note: unlike in Windows, file and directory names can only be letters, numbers, periods, hyphens, or underscores. Any other character or space must be preceded by a backslash character. This is known as escaping a character.

Run 1Password by typing:

./1Password.exe

This will bring up a Welcome to 1Password dialog box. Click on the I have used 1Password before button. Next navigate to your 1Password keychain in Dropbox.

Enter your 1Password master password. You should now see everything from your 1Password vault.

NEW: Lastly, navigate back to the 1Password application location and look for a file called Agile1pAgent.exe. This app must be running to enable the browser extension(s).

Run it by navigating to the 1Password directory and typing:

wine Agile1pAgent.exe

(Many thanks to Jonathan for providing this guidance in his comment to this post.)

End Notes

If you know of better/easier ways of doing this, please post a comment.

Thanks.

bash shell script to unlock encrypted disk in OS X 10.9

I finally buckled down and wrote my first bash shell script today.

I have been backing up my machines to two WD My Book 3 TB USB 3.0 Hard Drives.

I backup my hard drive to my server once a week. I also backup my hard drive to the WD USB drive every week.

I am swapping them out every week. I keep one drive at work at all times. This is keeping with the philosophy that at any one time I have

  • one backup on-site
  • one backup off-site

My only concern is that OS X leaves hard drives unencrypted by default. This means that anyone could plug an OS X-formatted drive into their Macintosh and read my files.

The solution: encryption.

On OS X, it is as easy as right-clicking on the hard drive in Finder and selecting ‘encrypt drive’. This is a good thing.

However, once I reconnect my USB drive I have to unlock the disk before I can make my backups. Complicating everything further, I divided the 3TB disk into four partitions: one for Cay’s computer, one for my boot drive, one for my /Users partition, and one that is on stand-by. So, every time I connect this drive I have to enter four different UUIDs to unlock the encrypted partitions.

Shell scripts to the rescue.

This weekend I spent some time learning to program UNIX shell scripts. I created a shell script to automate the unlocking of the USB drive upon connection. It was really fun investigating solutions for this. In the end I used many tools from the basic programming toolbox:

  • variables
  • redircts
  • pipes
  • regular expressions
  • arrays
  • and some more stuff

It was tons of fun and I learned a lot.

Here’s the code for the nerds out there.

#!/bin/bash

# =================================================================

# Copyright 2014 Eduardo Sanchez

# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.

# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.

# You should have received a copy of the GNU General Public License
# along with this program. If not, see .

# ==================================================================

# This script is used to unlock encrypted hdds in OS X 10.9

# First open Disk Utility.app to see which hdd are available

# 'locked' hdd will appear greyed-out

open -g /Applications/Utilities/Disk\ Utility.app

echo

# collect the UUIDs of all of the attached hdd

clear
echo "These are the attached hard drives:"

tempIFS=$IFS
IFS=$'\n'
diskutil cs list | grep -E "(Logical Volume) [A-F0-9]{8}-[A-F0-9]{4}-[A-F0-9]{4}-[A-F0-9]{4}-[A-F0-9]{12}" | cut -c 28-63 > hdd_UUIDs
uuid=($(cat hdd_UUIDs))
IFS=$tempIFS

echo

# collect the names of the Logical Volumes (disk names)

tempIFS=$IFS
IFS=$'\n'
diskutil cs list | grep "LV Name" | cut -c 36-70 > hdd_LV_Names
dname=($(cat hdd_LV_Names))
IFS=$tempIFS

# present the user with a listing of UUIDs and associated Logical Volume names

echo "0." ${uuid[0]} ${dname[0]}
echo "1." ${uuid[1]} ${dname[1]}
echo "2." ${uuid[2]} ${dname[2]}
echo "3." ${uuid[3]} ${dname[3]}
echo "4." ${uuid[4]} ${dname[4]}
echo

# ask the user which disk to unlock

echo Select the hdd to unlock
echo
echo

# gather the user's choice

read DISK

# DO NOT store the password as a variable!
# Keep passwords in a protected vault such as LastPass, Keypass, or 1Password

open -g /Applications/1Password\ 4.app

# unlock the disk selected by the user

diskutil cs unlockVolume ${uuid[$DISK]} -stdinpass

# User then pastes the password to complete the unlocking of the hdd

# TO DO List
#
# Figure out how to process the UUID and LV Name without creating
# an intermediate file
#
# Incorporate logic that will only present 'Locked' volumes
#
# this will do away with needing the Disk\ Utility.app open
#
# incorporate some crazy Keyboard Maestro kung-foo to auto run this
# script upon attaching an encrypted hdd