I’m testing a new version of the vimpress plugin

I have a unified way of supporting both python2 and python3 now. And the
plugin works great!

Advertisements

Tail probabilities of Gaussians

Tail probabilities for the Gaussian

There are many many tail probabilities for the Gaussian, and most use
some form of integration by parts. Duembgen’s paper on bounding
standard tail probabilities
has
several, for example.

Let
$$\Prob( X \geq t) = T(t) = \int_{t}^{\infty} \exp(-x^2/2) \frac{1}{\sqrt{2\pi}} dx$$
Two simple inequalities obtained by integration by parts are:
$$\begin{aligned}
\frac{1}{\sqrt{2\pi}} \exp(-t^2/2) \left(\frac{1}{t} – \frac{1}{t^3}\right) \leq T(t) \leq \frac{1}{t} \exp(-t^2/2) \end{aligned}$$
Fleshed out versions of these calculations can be found anywhere; see
https://mikespivey.wordpress.com/2011/10/21/normaltails/.
Essentially one simply introduces $x/t$ into the integrand to obtain an
upper bound. The integrand now has a simple anti-derivative.

Here is a better bound, using a slightly cooler trick of introducing a
new independent variable.

Introduce an independent Gaussian variable $Y$ and consider
$T(t) = \Prob( Y \in \R, X \geq t)$ again. This is bounded above by the
probability of the exterior of the circle: $$\begin{aligned}
\Prob( Y \in \R, X \geq t)
& \leq \Prob( X^2 + Y^2 \geq t^2)\
& = \frac1{2 \pi} \int_t^{\infty} r e^{-r^2/2} dr \
& = \frac1{2 \pi} e^{-t^2/2} \end{aligned}$$ Certainly
$\frac1{2\pi} \leq \frac12$; this is the bound that appears frequently
in papers: $$
\frac{1}{L(1 + t)} \exp\left( -\frac{t^2}{2}\right) \leq T(t) \leq \frac{1}{2} \exp\left( -\frac{t^2}{2} \right).
$$ I haven’t worked out the details of the lower bound, but I assume
it’s quite similar.

Using the iMac 27″ (late 2015) as an external monitor

I had an extra iMac 27″ (late 2011) that I wanted to use as a monitor
with my thinkpad that runs linux. This is quite easy to do and involves
something called Target display mode that the iMac can be put into.
I was concerned that this was a Mac-only mode and that it wouldn’t work
on linux.

There are two computers involved:

thinkpad:   drives the imac, runs linux
imac 27":   works as an external display, has os x installed and an apple keyboard connected to it. 
  1. First visit this page on apple’s website and determine
    what kind of connectors are supported in your imac. My thinkpad and
    imac both have a mini display port connector. I bought one off
    amazon for about \$15.
  2. Then install the arandr, and autorandr utilities on your linux box.
    I use the newer fork of autorandr written in
    python
    . arandr is a gui
    that makes things a little easier.
  3. On the linux box, run the command,
        while :; do xrandr | grep '\'; date '+%T'; sleep 2s; done
    

    This should display a messages that shows what monitors are
    connected

  4. Hit command + F2 on your imac to put it into target display mode.
    You should see new “connected” messages appearing on your linux box.
  5. At the same time, simultaneously run arandr, and hit activate on
    your the arandr menu. This will automatically enable the external
    monitor. You only have a window of about 5 seconds from when you hit
    command + F2 on your imac until you activate the display on your
    linux box. The alternative to using arandr is to have a command like
    the following entered into a terminal that you’re ready to hit enter
    on:

    xrandr --output eDP1 --mode 1920x1080 --output DP2-2 --mode 2560x1440
    

    The problem with the latter method is that it requires you to know
    the name that your display will be detected as. IN this case, it was
    detected as DP2-2. I suppose, a priori, it ought to be possible to
    determine what port the monitor is connected to, but I don’t know
    how.

  6. This should activate the imac as an external display. Now you want
    it to be automatically put into this display setting as soon as it
    is detected. This is where the autorandr script and its udev rules
    (that detect the external monitor being connected) come into play. I
    used

    autorandr -s imac-external
    

    Then if you’re on a systemd based distribution enable the
    autorandr.service and start it up with

    systemctl start autorandr
    systemctl enable autorandr
    

    both run as the root user. Make sure you’ve put the appropriate rule
    into

    /usr/lib/udev/rules.d/40-monitor-hotplug.rules
    

    Then reload your udev rules using

    udevadm control --reload-rules
    

Now your imac should be automatically detected and turned on when you 1)
turn on target display mode by hitting command + F2 and 2) connect
your laptop/desktop to the imac.

How to overcome incoming ssh firewalls through an intermediate relay ssh server

The way to forward ssh is as follows. I essentially read the openssh
cookbook here

https://en.wikibooks.org/wiki/OpenSSH/Cookbook/Tunnels

I have machine A (thinkpad), behind a firewall that cannot accept
incoming network connections. Then we have machine B (imac) that is also
behind a firewall that cannot accept incoming network connections. But
they can both connect to my home server. A simple schematic of the
allowed connections is as follows:

thinkpad --> shirl_dell_server
imac --> shirl_dell_server

I want to make a connection from

thinkpad --> imac

The way to do this is to use shirl_dell_server as a relay to
circumvent the firewall that prevents incoming connections on the
thinkpad and imac. For example, my University does this. The way to do
this is to establish one normal ssh tunnel and one reverse ssh tunnel.

  1. From thinkpad run
    ssh -fN -L 9000:localhost:9000 shirl_dell_server 
    

    This establishes a tunnel from local port 9000 (-L) to remote port
    9000 on shirl_dell_server. The -N allows you not run any command
    on the remote server — it usually runs the login shell — and the
    -f allows the process to drop to the background.

  2. From the imac run

    ssh -fN -R 9000:localhost:22 shirl_dell_server
    

    This establishs a reverse ssh tunnel from remote port 9000 (-R) to
    the localport 22 where an sshd server is listening for incoming
    connections.

  3. Then from thinkpad, run

    ssh -p 9000 localhost -l username
    

    This would get forwarded to the remote server’s 9000 port through
    the ssh tunnel. The remote server would forward the incoming traffic
    on 9000 to the imac’s port 22 through the reverse ssh tunnel.

This would then show you an ssh login prompt.

Note that you may have to enable to options

AllowTcpForwarding  yes
PermitTunnel        yes

Note that disabling AllowTcpForwarding does not improve security as the
sshd_config manpage says. Permitting tunnels, however, might decrease
security since it might allow you to circumvent other firewalls.

Silent failure of anacron on resume with systemd

Anacron fails silently on resume. It never runs the tasks its supposed
to run. My backups have been failing.

This is a a really annoying failure, and some features are being
implemented in systemd to fix this. See this debian
bugtracker
.

My solution for this is to simply have anacron be run every hour by
cron. I have in /etc/cron.d/anacron

# /etc/cron.d/anacron: crontab entries for the anacron package

SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

30 *    * * *   root    test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null

which simply runs it every hour now. This is a little annoying, but it’s
the best I can do. The debian bugtracker states that it’s been fixed in
a future release of systemd. I don’t really care because I’m using
Jessie. But Jessie is so old that apparently even critical bugs like
this don’t get fixed.

pass, passmenu and dmenu are pretty amazing together

I’ve switched to pass for managing my
passwords on my linux box, nexus phone and android tablet. It’s pretty
amazing.

Passwords are encrypted using pgp, and synchronized on my desktops using
dropbox. pass also has git support. So I push all my passwords to my
home ssh server using git. I can pull down these passwords to my android
phone and tablet using git packages and a pgp manager.

Setting pass up on android is a little painful. I use

  1. A pgp key management app.
  2. pass for
    android

To get my private pgp key into the app, I do something fairly insecure.
I transfer my key onto a dropbox encfs partition. Then I access this
encfs partition using another app called

BoxCryptor Classic

Then I also need to generate an ssh keypair so that pass for android has
access to my ssh server at home. I again copy the public key to dropbox
and have my server copy it there into the authorized users file.

Passmenu (bundled with pass) is especially powerful since I use dmenu to
launch programs.

Hibernate issues on Thinkpad T450s

Read updates after the main body for context.

UPDATE (May 22 2017) When I forced the firmware to use ucode version
17 for the iwlwifi driver, my network has been a lot more stable even on
the enterprise networks like my university network and eduroam.
Hibernate is pretty stable except for the following problem:

Close the lid and systemd-logind triggers suspend. Using the suspend hook,
the delayed hibernate script triggers a wake after 2 minutes. Then the
computer hibernates. However, I cannot awaken the computer (using the power
switch until I plug it in again!) Once I do so, the power switch works
normally and the computer resumes from hibernate.

One option is to manually handle the powerswitch using xfce power
manager or something. Here are some ideas:

  1. Disable systemd-logind handling of the lidswitch and try xfce4 power
    manager, and see if the original issues it had are resolved.
  2. Disable logind and xfce power manager (that I use primarily for
    dimming and locking the screen). Then, switch to acpid for hitting
    systemctl to handle the lidswitch

UPDATE (May 13 2017) I couldn’t track down the patches on the 4.4
kernel to apply it properly. So I reverted the ucode version that the
iwlwifi driver was using on the 4.10 kernel to see if this solves my
wifi instability issues.

dmesg | grep iwl

The 4.10 kernel appears to use

loaded firmware version 22.391740.0 op_mode iwlmvm

and the 4.4 kernel uses

loaded firmware version 17.352738.0 op_mode iwlmvm

I deleted the

/usr/lib/firmware/iwlwifi/*7265D-2[0-9].ucode

and now the 4.10 kernel seems to load an earlier firmware version.
Perhaps this fixes the instability on networks that use WPA2-Enterprise.

UPDATE (May 12 2017) I have wifi instability issues on the newer
kernels like 4.9 – 4.10 on the iwlwifi driver. I don’t know what
changed, but I frequently need to ifconfig down/up the interface or
rmmod/insmod the iwlwifi driver.

So I went back to the 4.4-lts kernel available on the AUR with the
4.4.64 patch set. Then I installed the following two patches:

  1. https://patchwork.kernel.org/patch/9208541/
  2. https://patchwork.kernel.org/patch/9202321/

One of the patches didn’t apply since they were meant for the 4.1.6
kernel, so I manually inserted some of the rejected patches. It appears
to work after a couple of tests. The patch appears to be merged with the
following commit number

406f992e4a372dafbe3c2cff7efbb2002a5c8ebd

It’s quite annoying that the iwlwifi driver is so unstable on the newer
kernels with my Intel 7265 card. I will update it if there is a problem.
Also, the unpatched 4.4.0 kernel also appears to work quite well, even
without the patches, but I haven’t tested it with more that 8GB memory
in used, which is what seems to create the problem.

UPDATE (May 09 2017) Intel rapid start (irst) craps out every once
in a while. After a few successful hibernates, it seems to go into a
boot loop of some sort, and then simply unsuccessfully reboots. So I’ve
disabled it, and moved to a newer kernel (4.9) that does not appear to
have hibernate issues. I’m using this delayed hibernate script on the
ArchWiki.

I’ve been having Hibernate issues on my thinkpad 450s. Apparently this
is quite common see this kernel bug
report
and has
existed in the newer kernels since version 3.x. So my solution was to
use the Intel Rapid Start technology. It was originally suggested on the
kernel bug report. It was quite easy to do.

  1. Make a partition that is greater than or equal to the amount of ram
    in your notebook.

    gdisk /dev/sda
    n
    

    Then, set partition name, start sector and size by following the
    prompts. Select 8400 (Intel Rapid Start) when prompted for a
    Hex Code or GUID. Hit w to write the partition table and q to
    quit.

  2. Go into the bios, and under power you’ll find the Intel Rapid Start
    technology. You can set deep sleep mode to set in after an hour of
    suspending – that’s what I’ve set mine too.

  3. Next time just suspend your thinkpad. Rapid Start works flawlessly.

The old hibernate issues have been resolved on newer kernels (4.7 and
above). But I always find hibernate a little buggy. For example, the
display frequently gets screwed up on my laptop and I have to restart X
or my display manager.