How to overcome incoming ssh firewalls through an intermediate relay ssh server

The way to forward ssh is as follows. I essentially read the openssh
cookbook here

I have machine A (thinkpad), behind a firewall that cannot accept
incoming network connections. Then we have machine B (imac) that is also
behind a firewall that cannot accept incoming network connections. But
they can both connect to my home server. A simple schematic of the
allowed connections is as follows:

thinkpad --> shirl_dell_server
imac --> shirl_dell_server

I want to make a connection from

thinkpad --> imac

The way to do this is to use shirl_dell_server as a relay to
circumvent the firewall that prevents incoming connections on the
thinkpad and imac. For example, my University does this. The way to do
this is to establish one normal ssh tunnel and one reverse ssh tunnel.

  1. From thinkpad run
    ssh -fN -L 9000:localhost:9000 shirl_dell_server 

    This establishes a tunnel from local port 9000 (-L) to remote port
    9000 on shirl_dell_server. The -N allows you not run any command
    on the remote server — it usually runs the login shell — and the
    -f allows the process to drop to the background.

  2. From the imac run

    ssh -fN -R 9000:localhost:22 shirl_dell_server

    This establishs a reverse ssh tunnel from remote port 9000 (-R) to
    the localport 22 where an sshd server is listening for incoming

  3. Then from thinkpad, run

    ssh -p 9000 localhost -l username

    This would get forwarded to the remote server’s 9000 port through
    the ssh tunnel. The remote server would forward the incoming traffic
    on 9000 to the imac’s port 22 through the reverse ssh tunnel.

This would then show you an ssh login prompt.

Note that you may have to enable to options

AllowTcpForwarding  yes
PermitTunnel        yes

Note that disabling AllowTcpForwarding does not improve security as the
sshd_config manpage says. Permitting tunnels, however, might decrease
security since it might allow you to circumvent other firewalls.

Silent failure of anacron on resume with systemd

Anacron fails silently on resume. It never runs the tasks its supposed
to run. My backups have been failing.

This is a a really annoying failure, and some features are being
implemented in systemd to fix this. See this debian

My solution for this is to simply have anacron be run every hour by
cron. I have in /etc/cron.d/anacron

# /etc/cron.d/anacron: crontab entries for the anacron package


30 *    * * *   root    test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null

which simply runs it every hour now. This is a little annoying, but it’s
the best I can do. The debian bugtracker states that it’s been fixed in
a future release of systemd. I don’t really care because I’m using
Jessie. But Jessie is so old that apparently even critical bugs like
this don’t get fixed.

pass, passmenu and dmenu are pretty amazing together

I’ve switched to pass for managing my
passwords on my linux box, nexus phone and android tablet. It’s pretty

Passwords are encrypted using pgp, and synchronized on my desktops using
dropbox. pass also has git support. So I push all my passwords to my
home ssh server using git. I can pull down these passwords to my android
phone and tablet using git packages and a pgp manager.

Setting pass up on android is a little painful. I use

  1. A pgp key management app.
  2. pass for

To get my private pgp key into the app, I do something fairly insecure.
I transfer my key onto a dropbox encfs partition. Then I access this
encfs partition using another app called

BoxCryptor Classic

Then I also need to generate an ssh keypair so that pass for android has
access to my ssh server at home. I again copy the public key to dropbox
and have my server copy it there into the authorized users file.

Passmenu (bundled with pass) is especially powerful since I use dmenu to
launch programs.

Hibernate issues on Thinkpad T450s

Read updates after the main body for context.

UPDATE (May 22 2017) When I forced the firmware to use ucode version
17 for the iwlwifi driver, my network has been a lot more stable even on
the enterprise networks like my university network and eduroam.
Hibernate is pretty stable except for the following problem:

Close the lid and systemd-logind triggers suspend. Using the suspend hook,
the delayed hibernate script triggers a wake after 2 minutes. Then the
computer hibernates. However, I cannot awaken the computer (using the power
switch until I plug it in again!) Once I do so, the power switch works
normally and the computer resumes from hibernate.

One option is to manually handle the powerswitch using xfce power
manager or something. Here are some ideas:

  1. Disable systemd-logind handling of the lidswitch and try xfce4 power
    manager, and see if the original issues it had are resolved.
  2. Disable logind and xfce power manager (that I use primarily for
    dimming and locking the screen). Then, switch to acpid for hitting
    systemctl to handle the lidswitch

UPDATE (May 13 2017) I couldn’t track down the patches on the 4.4
kernel to apply it properly. So I reverted the ucode version that the
iwlwifi driver was using on the 4.10 kernel to see if this solves my
wifi instability issues.

dmesg | grep iwl

The 4.10 kernel appears to use

loaded firmware version 22.391740.0 op_mode iwlmvm

and the 4.4 kernel uses

loaded firmware version 17.352738.0 op_mode iwlmvm

I deleted the


and now the 4.10 kernel seems to load an earlier firmware version.
Perhaps this fixes the instability on networks that use WPA2-Enterprise.

UPDATE (May 12 2017) I have wifi instability issues on the newer
kernels like 4.9 – 4.10 on the iwlwifi driver. I don’t know what
changed, but I frequently need to ifconfig down/up the interface or
rmmod/insmod the iwlwifi driver.

So I went back to the 4.4-lts kernel available on the AUR with the
4.4.64 patch set. Then I installed the following two patches:


One of the patches didn’t apply since they were meant for the 4.1.6
kernel, so I manually inserted some of the rejected patches. It appears
to work after a couple of tests. The patch appears to be merged with the
following commit number


It’s quite annoying that the iwlwifi driver is so unstable on the newer
kernels with my Intel 7265 card. I will update it if there is a problem.
Also, the unpatched 4.4.0 kernel also appears to work quite well, even
without the patches, but I haven’t tested it with more that 8GB memory
in used, which is what seems to create the problem.

UPDATE (May 09 2017) Intel rapid start (irst) craps out every once
in a while. After a few successful hibernates, it seems to go into a
boot loop of some sort, and then simply unsuccessfully reboots. So I’ve
disabled it, and moved to a newer kernel (4.9) that does not appear to
have hibernate issues. I’m using this delayed hibernate script on the

I’ve been having Hibernate issues on my thinkpad 450s. Apparently this
is quite common see this kernel bug
and has
existed in the newer kernels since version 3.x. So my solution was to
use the Intel Rapid Start technology. It was originally suggested on the
kernel bug report. It was quite easy to do.

  1. Make a partition that is greater than or equal to the amount of ram
    in your notebook.

    gdisk /dev/sda

    Then, set partition name, start sector and size by following the
    prompts. Select 8400 (Intel Rapid Start) when prompted for a
    Hex Code or GUID. Hit w to write the partition table and q to

  2. Go into the bios, and under power you’ll find the Intel Rapid Start
    technology. You can set deep sleep mode to set in after an hour of
    suspending – that’s what I’ve set mine too.

  3. Next time just suspend your thinkpad. Rapid Start works flawlessly.

The old hibernate issues have been resolved on newer kernels (4.7 and
above). But I always find hibernate a little buggy. For example, the
display frequently gets screwed up on my laptop and I have to restart X
or my display manager.

Exim with gmail as smarthost

I’ve been using exim with my google account to send email quite
successfully. I essentially followed the arch
It allows me to send mail from any of my machine on my local area
network through my server. This is quite convenient because a lot of
them run cron jobs, and it’s useful to a have an email relay server
running on the network. Comcast, my ISP, blocks smtp ports and so I
cannot run my own email server at home. So I setup an exim based relay
that accepts email from within my local area network and sends it using
my gmail address.

However, it stopped working without warning one day. My log
/var/log/exim/mainlog started being populated with messages like

    535-5.7.8 Username and Password not accepted. Learn more at
    535 5.7.8 v9sm11944329pfg.133 - gsmtp

This was apparently because I did not have two-factor authentication
enabled, and gmail had started flagging my application. So I enabled
two-factor authentication, and gave my it an app password. It started
working again.

Getting ibus to work for Tamil input

–\> I was trying to get tamil input to work on both my debian and arch
systems, since I wanted to use the tamil
to look up
words. There are two ways of doing this.

  1. Use xkb layouts. xkb supports tamil input natively, but they keymaps
    are a little weird. I spent some time trying to figure out the
    keymap it was using, but wasn’t very successful. I’m guessing it’s
    using the Tamil 99 layout, that can be found
  2. Input methods. This is the more powerful way of doing things,
    especially because there are multiple tamil layouts available and
    it’s easy to switch between them. If you use xkb, you’d typically
    have to restart X to install new layouts.

There are a bunch of input methods

  • scim
  • ibus
  • nimf (korean, chinese)
  • dasom (korean)
  • fcitx (popular in taiwan)
  • uim (japanese)

Of these, the scim and ibus are the most well known, and work well with
the m17n library. They’re largely the same, but ibus is the default in
many desktop environments. This
thinks that ibus’
interface is a little better. I think one can’t go wrong with either
one. I chose ibus.

Install the following packages

ibus-m17n, ibus-gtk, ibus-gtk3, ibus-qt4

There are ibus packages available for clutter too. For Arch Linux, I
followed their wiki.

You need to set a few environment variables:

export GTK_IM_MODULE=ibus
export XMODIFIERS=@im=ibus
export QT_IM_MODULE=ibus

On debian, it’s a bit easier since you first install the im-config
package and then run

im-config -n ibus

which will write an .xinputrc file. Debian has some dbus rules setup
that automatically set the GTK_IM_MODULE and other environment
variables. So it doesn’t appear to be necessary to insert the
environment variables into your .xinputrc. In any case, it doesn’t
seem to hurt whether you do it or not.

The Arch Wiki suggests inserting

ibus-daemon -drx

into my .xinputrc. This doesn’t work so well if you use a login or
display manager like lightdm. This is because the ibus daemon does not
like being started too early
. A symptom of this was that tamil input
worked inside my terminal emulator, but not inside firefox.

I use lightdm and i3, and what works for me is to insert
ibus-daemon -drx into my .i3/config file as

exec --no-startup-id ibus-daemon -drx

For other desktop environments like xfce or cinnamon, use the system
that allows you to specify ibus-daemon as a startup program.

I use phonetic tamil input and it’s absolutely amazing.

Remember to install tamil fonts! See this

Mathjax font errors in firefox

I started getting errors in firefox recently complaining about Math Processing Errors and problems loading imagefonts. It was sort of like this bug where a file called imagedata.js was not getting loaded.

Mathjax shouldn’t have to use imagefonts in any modern browser; mathjax was working perfectly in chrome for example. So what had happened was I’d unchecked the

Allow pages to choose their own fonts, instead of my selections above

under Preferences > Content > Advanced.

My firefox version is

Mozilla Firefox 45.0.1

It’s also interesting to note that the usual packages in ArchLinux and Ubuntu don’t package MathJax image fonts by default. These are however available in their github repository.