bitstream.io/2021-12-26T08:21:50-08:00Adding a `git hub` alias2021-12-26T08:21:50-08:002021-12-26T08:21:50-08:00rajtag:None,2021-12-26:/adding-a-git-hub-alias.html<p>This git alias will open a web browser and take you to a project's github page when you
type <code>git hub</code> in a checked-out project directory.</p>
<p>For OS X:</p>
<div class="highlight"><pre><span></span><code>git config --global alias.hub <span class="s1">'!open $(echo $(git config --get remote.origin.url) | perl -pe "s|git\@github\.com:|https://github …</span></code></pre></div><p>This git alias will open a web browser and take you to a project's github page when you
type <code>git hub</code> in a checked-out project directory.</p>
<p>For OS X:</p>
<div class="highlight"><pre><span></span><code>git config --global alias.hub <span class="s1">'!open $(echo $(git config --get remote.origin.url) | perl -pe "s|git\@github\.com:|https://github.com/|")'</span>
</code></pre></div>
<p>For Linux:</p>
<div class="highlight"><pre><span></span><code>git config --global alias.hub <span class="s1">'!xdg-open $(echo $(git config --get remote.origin.url) |perl -pe "s|git\@github\.com:|https://github.com/|")'</span>
</code></pre></div>Using Logspout and Papertrail to collect logs from containers running in Amazon ECS2016-03-01T00:00:00-08:002021-12-26T08:21:50-08:00rajtag:None,2016-03-01:/using-logspout-and-papertrail-to-collect-logs-from-containers-running-in-amazon-ecs.html<p>Amazon <a href="https://aws.amazon.com/ecs/">ECS</a> is a container management service that makes it
easy to deploy Docker containers on AWS. However, once you scale to many containers running
on many EC2 nodes, you will need a centralized service to collect and analyze docker logs.</p>
<p>Centralized log collection is quite easy to set up …</p><p>Amazon <a href="https://aws.amazon.com/ecs/">ECS</a> is a container management service that makes it
easy to deploy Docker containers on AWS. However, once you scale to many containers running
on many EC2 nodes, you will need a centralized service to collect and analyze docker logs.</p>
<p>Centralized log collection is quite easy to set up with the <a href="https://github.com/gliderlabs/logspout">Logspout</a>
Docker container and the <a href="https://papertrailapp.com">Papertrail</a> log management service.</p>
<p>Logspout is a docker container that attaches to all the other docker containers running
on a host and forwards the docker logs somewhere.</p>
<p>Papertrail is a log aggregator that provides a remote syslog endpoint. It has a nice
web console and a fast search engine that lets you tail logs in realtime and
filter by syslog priority, machine name, program name, or search string.
Papertrail also has integrations for sending alerts to many services, such as PagerDuty,
Amazon SNS, and Slack.</p>
<p>Log aggregation for ECS is easily achieved by running one Logspout container on
each EC2 node in your ECS cluster, which will forward logs for all docker containers
running on the EC2 node to Papertrail using syslog. This lets us see and analyze
logs in a centralized place, but also leaves the logs on the EC2 nodes in case
something goes wrong with log forwarding and we need to analyze the logs
directly on the host.</p>
<p>We can deploy Logspout in ECS using the standard ECS scheduler, but by default, we can't
tell ECS to schedule exactly one Logspout container per EC2 node. However, if we launch
the same number of Logspout containers as EC2 nodes in the cluster, and we tell ECS to
attach an unused port to Logspout, then the scheduler will launch exactly one Logspout
container per EC2 node.</p>
<p>You can use the <code>boto</code> python library with the code below to create a Task Definition and
an ECS Service for Logspout in your ECS cluster:</p>
<div class="highlight"><pre><span></span><code><span class="c1"># Create ECS Task Definition</span>
<span class="n">task_def</span> <span class="o">=</span> <span class="p">{</span>
<span class="s1">'image'</span><span class="p">:</span> <span class="s1">'gliderlabs/logspout'</span><span class="p">,</span>
<span class="s1">'name'</span><span class="p">:</span> <span class="s1">'logspout'</span><span class="p">,</span>
<span class="s1">'essential'</span><span class="p">:</span> <span class="kc">True</span><span class="p">,</span>
<span class="s1">'memory'</span><span class="p">:</span> <span class="mi">128</span><span class="p">,</span>
<span class="s1">'cpu'</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
<span class="s1">'portMappings'</span><span class="p">:</span> <span class="p">[{</span>
<span class="s1">'containerPort'</span><span class="p">:</span> <span class="mi">1234</span><span class="p">,</span> <span class="c1"># trick ECS into only scheduling one container per node</span>
<span class="s1">'hostPort'</span><span class="p">:</span> <span class="mi">1234</span><span class="p">,</span> <span class="c1"># by mapping a dummy port to our logspout container</span>
<span class="s1">'protocol'</span><span class="p">:</span> <span class="s1">'tcp'</span>
<span class="p">}],</span>
<span class="s1">'mountPoints'</span><span class="p">:</span> <span class="p">[{</span>
<span class="s1">'containerPath'</span><span class="p">:</span> <span class="s1">'/tmp/docker.sock'</span><span class="p">,</span> <span class="c1"># logspout uses the docker unix socket</span>
<span class="s1">'sourceVolume'</span><span class="p">:</span> <span class="s1">'dockersock'</span> <span class="c1"># to gather logs from the other containers</span>
<span class="p">}],</span>
<span class="s1">'command'</span><span class="p">:</span> <span class="p">[</span><span class="s2">"syslog://HOST.papertrailapp.com:PORT"</span><span class="p">]</span> <span class="c1">#replace HOST and PORT with your account's config</span>
<span class="p">}</span>
<span class="n">volume</span> <span class="o">=</span> <span class="p">{</span>
<span class="s1">'host'</span><span class="p">:</span> <span class="p">{</span><span class="s1">'sourcePath'</span><span class="p">:</span> <span class="s1">'/var/run/docker.sock'</span><span class="p">},</span>
<span class="s1">'name'</span><span class="p">:</span> <span class="s1">'dockersock'</span>
<span class="p">}</span>
<span class="n">client</span> <span class="o">=</span> <span class="n">boto3</span><span class="o">.</span><span class="n">client</span><span class="p">(</span><span class="s1">'ecs'</span><span class="p">,</span> <span class="n">region_name</span><span class="o">=</span><span class="s1">'xxx'</span><span class="p">)</span>
<span class="n">client</span><span class="o">.</span><span class="n">register_task_definition</span><span class="p">(</span>
<span class="n">family</span> <span class="o">=</span> <span class="s1">'my-task-def'</span><span class="p">,</span>
<span class="n">containerDefinitions</span> <span class="o">=</span> <span class="p">[</span><span class="n">task_def</span><span class="p">],</span>
<span class="n">volumes</span> <span class="o">=</span> <span class="p">[</span><span class="n">volume</span><span class="p">]</span>
<span class="p">)</span>
<span class="c1"># Create ECS Service</span>
<span class="n">client</span><span class="o">.</span><span class="n">create_service</span><span class="p">(</span>
<span class="n">desiredCount</span> <span class="o">=</span> <span class="n">N</span><span class="p">,</span> <span class="c1">#replace with number of EC2 nodes in your cluster</span>
<span class="n">cluster</span> <span class="o">=</span> <span class="s1">'my-cluster-name'</span><span class="p">,</span>
<span class="n">serviceName</span> <span class="o">=</span> <span class="s1">'logspout-service'</span><span class="p">,</span>
<span class="n">taskDefinition</span> <span class="o">=</span> <span class="s1">'my-task-def'</span>
<span class="p">)</span>
</code></pre></div>
<p>After ECS launches the logspout containers, your docker logs will start appearing in Papertrail.</p>Rebooting a linux node, even if shutdown hangs on a stuck device.2015-01-14T00:00:00-08:002021-12-26T08:21:50-08:00rajtag:None,2015-01-14:/rebooting-a-linux-node-even-if-shutdown-hangs-on-a-stuck-device.html<p>Sam showed me how to force a linux box to reboot, even if shutdown gets wedged on a stuck
device. In our case, we had a devicemanager device that had hung (which you could see by
running <code>iostat -xm 1</code>)</p>
<p>We use <code>softdog</code>, the nonhardware assisted watchdog driver to hard …</p><p>Sam showed me how to force a linux box to reboot, even if shutdown gets wedged on a stuck
device. In our case, we had a devicemanager device that had hung (which you could see by
running <code>iostat -xm 1</code>)</p>
<p>We use <code>softdog</code>, the nonhardware assisted watchdog driver to hard reset the
machine if needed. <code>softdog</code> will hard reset the machine if it doesn't receive
data every 60 seconds, so we open <code>/dev/watchdog</code> for write and then issue a
<code>shutdown -r</code>:</p>
<div class="highlight"><pre><span></span><code>modprobe softdog
sudo bash -c <span class="s2">"cat > /dev/watchdog"</span> <span class="p">&</span>
shutdown -r now
</code></pre></div>
<p>Thanks, Sam!</p>Compiling and installing gphoto2014-12-12T14:45:00-08:002021-12-26T08:21:50-08:00rajtag:None,2014-12-12:/compiling-and-installing-gphoto.html<p>Here are instructions on how to compile and install the latest version of gphoto, from SVN.</p>
<div class="highlight"><pre><span></span><code><span class="c1"># install dependencies (for Ubuntu 14.04)</span>
$ sudo apt-get install automake autopoint gettext libtool libusb-dev libpopt-dev subversion
$ mkdir gphoto
$ <span class="nb">cd</span> gphoto
<span class="c1"># compile libphoto2 and install into /usr/local/lib</span>
$ svn co https://svn.code.sf …</code></pre></div><p>Here are instructions on how to compile and install the latest version of gphoto, from SVN.</p>
<div class="highlight"><pre><span></span><code><span class="c1"># install dependencies (for Ubuntu 14.04)</span>
$ sudo apt-get install automake autopoint gettext libtool libusb-dev libpopt-dev subversion
$ mkdir gphoto
$ <span class="nb">cd</span> gphoto
<span class="c1"># compile libphoto2 and install into /usr/local/lib</span>
$ svn co https://svn.code.sf.net/p/gphoto/code/trunk/libgphoto2
$ <span class="nb">cd</span> libgphoto2/
$ autoreconf --install --symlink
$ ./configure --prefix<span class="o">=</span>/usr/local
$ make
$ sudo make install
<span class="c1"># compile gphoto2 and install into /usr/local/bin</span>
$ <span class="nb">cd</span> ..
$ svn co https://svn.code.sf.net/p/gphoto/code/trunk/gphoto2
$ <span class="nb">cd</span> gphoto2/
$ autoreconf --install --symlink
$ <span class="nv">PKG_CONFIG_PATH</span><span class="o">=</span>/usr/local/lib/pkgconfig ./configure --prefix<span class="o">=</span>/usr/local
$ make
$ sudo make install
</code></pre></div>
<p>Be sure to set LD_LIBRARY_PATH so that gphoto can find the correct version of libgphoto2
and libgphoto2_port:</p>
<div class="highlight"><pre><span></span><code>$ <span class="nv">LD_LIBRARY_PATH</span><span class="o">=</span>/usr/local/lib /usr/local/bin/gphoto2 --version
gphoto2 <span class="m">2</span>.5.5.1
Copyright <span class="o">(</span>c<span class="o">)</span> <span class="m">2000</span>-2014 Lutz Mueller and others
gphoto2 comes with NO WARRANTY, to the extent permitted by law. You may
redistribute copies of gphoto2 under the terms of the GNU General Public
License. For more information about these matters, see the files named COPYING.
This version of gphoto2 is using the following software versions and options:
gphoto2 <span class="m">2</span>.5.5.1 gcc, popt<span class="o">(</span>m<span class="o">)</span>, exif, no cdk, no aa, jpeg, no readline
libgphoto2 <span class="m">2</span>.5.5.3 all camlibs, gcc, ltdl, EXIF
libgphoto2_port <span class="m">0</span>.12.0 gcc, ltdl, USB, serial without locking
</code></pre></div>
<p>These instructions were tested on Ubuntu 14.04, and will compile all camera drivers. </p>Packaging and Distributing a Kivy application on Linux2014-08-16T00:00:00-07:002021-12-26T08:21:50-08:00rajtag:None,2014-08-16:/packaging-and-distributing-a-kivy-application-on-linux.html<p>The <a href="http://kivy.org">Kivy library</a> can be used to create cross-platform desktop and
mobile apps that can be distributed on Linux, OS X, Windows, iOS, and Android. Packaging
Kivy apps on Linux is not well-supported. Here is how to get it to work:</p>
<h2>1. Use a Vagrantfile to bootstrap a development environment …</h2><p>The <a href="http://kivy.org">Kivy library</a> can be used to create cross-platform desktop and
mobile apps that can be distributed on Linux, OS X, Windows, iOS, and Android. Packaging
Kivy apps on Linux is not well-supported. Here is how to get it to work:</p>
<h2>1. Use a Vagrantfile to bootstrap a development environment</h2>
<p>There are a lot of dependencies for kivy development, and a lot of different ways to
install them. In order to help you get started, I made a Vagrantfile to set up a
Kivy dev environment.</p>
<p>The Vagrantfile installs <code>python</code>, <code>kivy</code>, and <code>pyinstaller</code> in an Ubuntu VM, and then
packages a kivy example app into a <code>.deb</code>. You end up with a double-clickable application
that works like a regular linux desktop app.</p>
<p>The Vagrantfile and instructions on how to use it are here:
<a href="https://github.com/rajbot/kivy_pyinstaller_linux_example">https://github.com/rajbot/kivy_pyinstaller_linux_example</a></p>
<p><br/></p>
<h2>2. Install Kivy using <code>pip</code>, and not from the provided PPA</h2>
<p>If you don't use the Vagrantfile above and want to install Kivy yourself, do not
install it using the PPA. This is because we are going to use PyInstaller to
create a linux executable, and we will need the PyInstaller hooks from the <code>kivy.tools</code>
package, but the PPA does not include <code>kivy.tools</code>.</p>
<p>This <a href="https://github.com/rajbot/kivy_pyinstaller_linux_example/blob/master/bootstrap.sh">bash script</a>
will show you how to install Kivy in a virtualenv so that you can use the PyInstaller hooks.</p>
<p><br/></p>
<h2>3. Strip system libraries from the pyinstaller executable to ensure your app is relocatable</h2>
<p>If you follow the setup instructions above, you will end up an executable that works only
on the machine it was built. If you try to copy it to another linux box, kivy will often
segfault on startup with this error message:</p>
<div class="highlight"><pre><span></span><code>Fatal Python Error: (pygame parachute) Segmentation Fault
</code></pre></div>
<p>To ensure the executable can run on as many different flavors of linux as possible, we
are going to strip out all binaries provided by system packages. We will distribute the
application in a <code>.deb</code> file and and use <code>.deb</code> dependencies to ensure required libraries
are installed on the target machine.</p>
<p>Here is a copy of a PyInstaller <code>.spec</code> file that installs Kivy hooks and strips out all
binaries that <code>dpkg -S</code> finds in a system-installed library:</p>
<div class="highlight"><pre><span></span><code><span class="c1"># -*- mode: python -*-</span>
<span class="kn">from</span> <span class="nn">kivy.tools.packaging.pyinstaller_hooks</span> <span class="kn">import</span> <span class="n">install_hooks</span>
<span class="n">install_hooks</span><span class="p">(</span><span class="nb">globals</span><span class="p">())</span>
<span class="k">def</span> <span class="nf">filter_binaries</span><span class="p">(</span><span class="n">all_binaries</span><span class="p">):</span>
<span class="sd">'''Exclude binaries provided by system packages, and rely on .deb dependencies</span>
<span class="sd"> to ensure these binaries are available on the target machine.</span>
<span class="sd"> We need to remove OpenGL-related libraries so we can distribute the executable</span>
<span class="sd"> to other linux machines that might have different graphics hardware. If you</span>
<span class="sd"> bundle system libraries, your application might crash when run on a different</span>
<span class="sd"> machine with the following error during kivy startup:</span>
<span class="sd"> Fatal Python Error: (pygame parachute) Segmentation Fault</span>
<span class="sd"> If we strip all libraries, then PIL might not be able to find the correct _imaging</span>
<span class="sd"> module, even if the `python-image` package has been installed on the system. The</span>
<span class="sd"> easy way to fix this is to not filter binaries from the python-imaging package.</span>
<span class="sd"> We will strip out all binaries, except libpython2.7, which is required for the</span>
<span class="sd"> pyinstaller-frozen executable to work, and any of the python-* packages.</span>
<span class="sd"> '''</span>
<span class="nb">print</span> <span class="s1">'Excluding system libraries'</span>
<span class="kn">import</span> <span class="nn">subprocess</span>
<span class="n">excluded_pkgs</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span>
<span class="n">excluded_files</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span>
<span class="n">whitelist_prefixes</span> <span class="o">=</span> <span class="p">(</span><span class="s1">'libpython2.7'</span><span class="p">,</span> <span class="s1">'python-'</span><span class="p">)</span>
<span class="n">binaries</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">all_binaries</span><span class="p">:</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">output</span> <span class="o">=</span> <span class="n">subprocess</span><span class="o">.</span><span class="n">check_output</span><span class="p">([</span><span class="s1">'dpkg'</span><span class="p">,</span> <span class="s1">'-S'</span><span class="p">,</span> <span class="n">b</span><span class="p">[</span><span class="mi">1</span><span class="p">]],</span> <span class="n">stderr</span><span class="o">=</span><span class="nb">open</span><span class="p">(</span><span class="s1">'/dev/null'</span><span class="p">))</span>
<span class="n">p</span><span class="p">,</span> <span class="n">path</span> <span class="o">=</span> <span class="n">output</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">':'</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">p</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="n">whitelist_prefixes</span><span class="p">):</span>
<span class="n">excluded_pkgs</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">p</span><span class="p">)</span>
<span class="n">excluded_files</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">b</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
<span class="nb">print</span> <span class="s1">' excluding </span><span class="si">{f}</span><span class="s1"> from package </span><span class="si">{p}</span><span class="s1">'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">f</span><span class="o">=</span><span class="n">b</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">p</span><span class="o">=</span><span class="n">p</span><span class="p">)</span>
<span class="k">except</span> <span class="ne">Exception</span><span class="p">:</span>
<span class="k">pass</span>
<span class="nb">print</span> <span class="s1">'Your exe will depend on the following packages:'</span>
<span class="nb">print</span> <span class="n">excluded_pkgs</span>
<span class="n">inc_libs</span> <span class="o">=</span> <span class="nb">set</span><span class="p">([</span><span class="s1">'libpython2.7.so.1.0'</span><span class="p">])</span>
<span class="n">binaries</span> <span class="o">=</span> <span class="p">[</span><span class="n">x</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">all_binaries</span> <span class="k">if</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">excluded_files</span><span class="p">]</span>
<span class="k">return</span> <span class="n">binaries</span>
<span class="n">a</span> <span class="o">=</span> <span class="n">Analysis</span><span class="p">([</span><span class="s1">'scribe.py'</span><span class="p">],</span>
<span class="n">pathex</span><span class="o">=</span><span class="p">[</span><span class="s1">'.'</span><span class="p">],</span>
<span class="n">hiddenimports</span><span class="o">=</span><span class="p">[],</span>
<span class="p">)</span>
<span class="n">pyz</span> <span class="o">=</span> <span class="n">PYZ</span><span class="p">(</span><span class="n">a</span><span class="o">.</span><span class="n">pure</span><span class="p">)</span>
<span class="n">binaries</span> <span class="o">=</span> <span class="n">filter_binaries</span><span class="p">(</span><span class="n">a</span><span class="o">.</span><span class="n">binaries</span><span class="p">)</span>
<span class="n">exe</span> <span class="o">=</span> <span class="n">EXE</span><span class="p">(</span><span class="n">pyz</span><span class="p">,</span>
<span class="p">[(</span><span class="s1">'scribe.kv'</span><span class="p">,</span> <span class="s1">'scribe.kv'</span><span class="p">,</span> <span class="s1">'DATA'</span><span class="p">)],</span>
<span class="n">a</span><span class="o">.</span><span class="n">scripts</span><span class="p">,</span>
<span class="n">binaries</span><span class="p">,</span> <span class="c1">#a.binaries,</span>
<span class="n">a</span><span class="o">.</span><span class="n">zipfiles</span><span class="p">,</span>
<span class="n">a</span><span class="o">.</span><span class="n">datas</span><span class="p">,</span>
<span class="n">name</span><span class="o">=</span><span class="s1">'ia-scribe'</span><span class="p">,</span>
<span class="n">debug</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
<span class="n">strip</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span>
<span class="n">upx</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
<span class="n">console</span><span class="o">=</span><span class="kc">False</span> <span class="p">)</span>
</code></pre></div>
<p><br/></p>
<h2>4. Package your executable in a <code>.deb</code> file</h2>
<p>Once you make an executable, you can give it a nice icon and Ubuntu <code>.desktop</code>
file and package it in a <code>.deb</code> for distribution. Steps to create the <code>.deb</code> can
be found in <a href="https://github.com/rajbot/kivy_pyinstaller_linux_example/blob/master/bootstrap.sh">bootstrap.sh</a>,
from step #1 above. At minimum, your <code>.deb</code> package should:</p>
<ul>
<li>Install your application in a binary directory, such as <code>/usr/local/bin/my-app</code></li>
<li>Install an icon in <code>/usr/share/pixmaps/my-app.png</code></li>
<li>Install a <code>.desktop</code> file in <code>/usr/share/applications/my-app.desktop</code></li>
</ul>
<p>Debian version numbers are in the form {major}.{minor}-{patchlevel}. To make the <code>.deb</code>,
first create the directory structure below:</p>
<div class="highlight"><pre><span></span><code>- my-app_1.0-1
- DEBIAN
- control
- usr
- local
- bin
- my-app
- share
- applications
- my-app.desktop
- pixmaps
- my-app.png
</code></pre></div>
<p>Now you can package your app by typing <code>dpkg-deb --build myapp_1.0-0</code>. You can then
install the resulting package by typing <code>sudo dpkg -i myapp_1.0-0.deb</code>.</p>
<p>The <code>DEBIAN/control</code> file should look like this:</p>
<div class="highlight"><pre><span></span><code><span class="n">Source</span><span class="o">:</span> <span class="n">my</span><span class="o">-</span><span class="n">app</span>
<span class="n">Priority</span><span class="o">:</span> <span class="n">extra</span>
<span class="n">Maintainer</span><span class="o">:</span> <span class="n">raj</span> <span class="o"><</span><span class="n">raj</span><span class="err">@</span><span class="n">unknown</span><span class="o">></span>
<span class="n">Build</span><span class="o">-</span><span class="n">Depends</span><span class="o">:</span> <span class="n">debhelper</span> <span class="o">(>=</span> <span class="mf">8.0</span><span class="o">.</span><span class="mi">0</span><span class="o">)</span>
<span class="n">Standards</span><span class="o">-</span><span class="n">Version</span><span class="o">:</span> <span class="mf">3.9</span><span class="o">.</span><span class="mi">2</span>
<span class="n">Package</span><span class="o">:</span> <span class="n">my</span><span class="o">-</span><span class="n">app</span>
<span class="n">Version</span><span class="o">:</span> <span class="mf">1.0</span><span class="o">-</span><span class="mi">0</span>
<span class="n">Architecture</span><span class="o">:</span> <span class="n">i386</span>
<span class="n">Description</span><span class="o">:</span> <span class="n">Should</span> <span class="n">description</span>
<span class="n">Long</span> <span class="n">description</span> <span class="n">string</span> <span class="o">(</span><span class="n">starts</span> <span class="k">with</span> <span class="n">a</span> <span class="n">whitespace</span><span class="o">)</span>
</code></pre></div>
<p>To give your executable a first-class Ubuntu application, you will need to create a
<code>.desktop</code> file, which will tell Ubuntu about its icon, version, and name. <code>my-app.desktop</code>
should look like this:</p>
<div class="highlight"><pre><span></span><code><span class="k">[Desktop Entry]</span>
<span class="na">Version</span><span class="o">=</span><span class="s">1.0</span>
<span class="na">Name</span><span class="o">=</span><span class="s">My Application</span>
<span class="na">Comment</span><span class="o">=</span><span class="s">Example App</span>
<span class="na">Exec</span><span class="o">=</span><span class="s">/usr/local/bin/my-app</span>
<span class="na">Icon</span><span class="o">=</span><span class="s">my-app</span>
<span class="na">Type</span><span class="o">=</span><span class="s">Application</span>
<span class="na">Categories</span><span class="o">=</span><span class="s">Utility;Application;</span>
</code></pre></div>
<p>Note that <code>Version</code> above refers to the version of the <code>.desktop</code> format (and
not the version of the app), and should always be "1.0". The <code>Icon</code> entry does
not need a full path or extension. Ubuntu will look for your icon in
<code>/usr/share/pixmaps</code>.</p>
<p><br/></p>
<h2>5. Set up a signed trivial APT repository to distribute your <code>.deb</code></h2>
<p>To distribute your <code>.deb</code> file to end users, you will want to set up an APT
repository, which must be signed with a GPG key if you want to allow for
programatic installation or automatic updates.</p>
<p>Instructions for <a href="/creating-a-trivial-signed-apt-repository.html">setting up a signed trivial repo are provided here</a>.</p>Creating a Trivial, Signed APT Repository2014-08-13T00:00:00-07:002021-12-26T08:21:50-08:00rajtag:None,2014-08-13:/creating-a-trivial-signed-apt-repository.html<p>Setting up an APT repository is a reasonable way to distribute <code>.deb</code>
packages to end users. You will want to create a signed repository, so
that users can perform automated updates of your software (e.g. using
<code>apt-get install -y</code>).</p>
<p>There are <a href="https://wiki.debian.org/HowToSetupADebianRepository#APT_Archive_Types">two kinds of repositories</a>,
"official" and "trivial". Trivial …</p><p>Setting up an APT repository is a reasonable way to distribute <code>.deb</code>
packages to end users. You will want to create a signed repository, so
that users can perform automated updates of your software (e.g. using
<code>apt-get install -y</code>).</p>
<p>There are <a href="https://wiki.debian.org/HowToSetupADebianRepository#APT_Archive_Types">two kinds of repositories</a>,
"official" and "trivial". Trivial repositories are easier to set up than
the official variety, but setup of <em>signed</em> trivial repositories is not
well-documented. Here are instructions on how to set up your own repo:</p>
<p>Prerequisites:</p>
<ul>
<li>A <code>.deb</code> file, unsigned</li>
<li>A machine with <code>gpg</code> installed</li>
<li>RSA gpg keys created using <code>gpg --gen-key</code></li>
</ul>
<p>The repository is a simply a directory hierarchy. It can be local (e.g. on a usb stick) or
published to a web server. Here is what it should look like when we are done:</p>
<div class="highlight"><pre><span></span><code>- apt-repo/
- foo.deb
- Packages
- Release
- Release.gpg
</code></pre></div>
<p>Start by creating the hierarchy above with empty directories. Place <code>foo.deb</code> in the
<code>apt-repo</code> directory, and then follow these instructions to create the signed files:</p>
<div class="highlight"><pre><span></span><code><span class="c1">#sign the .deb file</span>
<span class="nb">cd</span> apt-repo
dpkg-sig --sign builder foo.deb
<span class="nb">cd</span> ..
<span class="c1">#create the Packages.gz file</span>
<span class="nb">cd</span> apt-repo
apt-ftparchive packages . > Packages
<span class="nb">cd</span> ..
<span class="c1">#create the Release file</span>
apt-ftparchive release apt-repo > apt-repo/Release
<span class="c1">#create a detached ascii signature of the Release file</span>
gpg --armor --sign --detach-sign apt-repo/Release.gpg apt-repo/Release
</code></pre></div>
<p>Now you can upload the <code>apt-repo</code> directory to a webserver. If the repo
path is <code>http://example.com/foo/apt-repo</code>, then end users will have to
add a file named <code>my-repo.list</code> to the <code>/etc/apt/sources.list.d</code> directory
that contains this line:</p>
<div class="highlight"><pre><span></span><code>deb http://example.com/foo/apt-repo /
</code></pre></div>
<p>You now need to distribute your public key to end users, who will need to add it to their
apt keychain. To export the public key to a text file:</p>
<div class="highlight"><pre><span></span><code>gpg --armor --export foo@example.com --output foo-public-key
</code></pre></div>
<p>Then upload <code>foo-public-key</code> to a webserver. End users can install the key like so:</p>
<div class="highlight"><pre><span></span><code>curl http://example.com/foo-public-key <span class="p">|</span> sudo apt-key add -
</code></pre></div>
<p>Once the key has been added, end users can install your software using <code>apt</code>:</p>
<div class="highlight"><pre><span></span><code>sudo apt-get update
sudo apt-get install foo
</code></pre></div>OS X and Ubuntu Codenames2014-08-13T00:00:00-07:002021-12-26T08:21:50-08:00rajtag:None,2014-08-13:/os-x-and-ubuntu-codenames.html<style type="text/css">
tbody td {font-size: 75%;}
</style>
<div class="row">
<div class="col-md-6">
<h2>Ubuntu</h2>
<table class="table-striped">
<tbody>
<tr><td>Ubuntu 21.10</td><td>Impish Indri</td></tr>
<tr><td>Ubuntu 21.04</td><td>Hirsute Hippo</td></tr>
<tr><td>Ubuntu 20.10</td><td>Groovy Gorilla</td></tr>
<tr><td>Ubuntu 20.04 LTS</td><td>Focal Fossa</td></tr>
<tr><td>Ubuntu 19.10</td><td>Eoan Ermine</td></tr>
<tr><td>Ubuntu 19.04</td><td>Disco Dingo</td></tr>
<tr><td>Ubuntu 18.10</td><td>Cosmic Cuttlefish</td></tr>
<tr><td>Ubuntu 18.04 LTS</td><td>Bionic Beaver</td></tr>
<tr><td>Ubuntu 17.10 …</td></tr></tbody></table></div></div><style type="text/css">
tbody td {font-size: 75%;}
</style>
<div class="row">
<div class="col-md-6">
<h2>Ubuntu</h2>
<table class="table-striped">
<tbody>
<tr><td>Ubuntu 21.10</td><td>Impish Indri</td></tr>
<tr><td>Ubuntu 21.04</td><td>Hirsute Hippo</td></tr>
<tr><td>Ubuntu 20.10</td><td>Groovy Gorilla</td></tr>
<tr><td>Ubuntu 20.04 LTS</td><td>Focal Fossa</td></tr>
<tr><td>Ubuntu 19.10</td><td>Eoan Ermine</td></tr>
<tr><td>Ubuntu 19.04</td><td>Disco Dingo</td></tr>
<tr><td>Ubuntu 18.10</td><td>Cosmic Cuttlefish</td></tr>
<tr><td>Ubuntu 18.04 LTS</td><td>Bionic Beaver</td></tr>
<tr><td>Ubuntu 17.10</td><td>Artful Aardvark</td></tr>
<tr><td>Ubuntu 17.04</td><td>Zesty Zapus</td></tr>
<tr><td>Ubuntu 16.10</td><td>Yakkety Yak</td></tr>
<tr><td>Ubuntu 16.04 LTS</td><td>Xenial Xerus</td></tr>
<tr><td>Ubuntu 15.10</td><td>Wily Werewolf</td></tr>
<tr><td>Ubuntu 15.04</td><td>Vivid Vervet</td></tr>
<tr><td>Ubuntu 14.10</td><td>Utopic Unicorn</td></tr>
<tr><td>Ubuntu 14.04 LTS</td><td>Trusty Tahr</td></tr>
<tr><td>Ubuntu 13.10</td><td>Saucy Salamander</td></tr>
<tr><td>Ubuntu 13.04</td><td>Raring Ringtail</td></tr>
<tr><td>Ubuntu 12.10</td><td>Quantal Quetzal</td></tr>
<tr><td>Ubuntu 12.04 LTS</td><td>Precise Pangolin</td></tr>
<tr><td>Ubuntu 11.10</td><td>Oneiric Ocelot</td></tr>
<tr><td>Ubuntu 11.04</td><td>Natty Narwhal</td></tr>
<tr><td>Ubuntu 10.10</td><td>Maverick Meerkat</td></tr>
<tr><td>Ubuntu 10.04 LTS</td><td>Lucid Lynx</td></tr>
<tr><td>Ubuntu 9.10</td><td>Karmic Koala</td></tr>
<tr><td>Ubuntu 9.04</td><td>Jaunty Jackalope</td></tr>
<tr><td>Ubuntu 8.10</td><td>Intrepid Ibex</td></tr>
<tr><td>Ubuntu 8.04 LTS</td><td>Hardy Heron</td></tr>
<tr><td>Ubuntu 7.10</td><td>Gutsy Gibbon</td></tr>
<tr><td>Ubuntu 7.04</td><td>Feisty Fawn</td></tr>
<tr><td>Ubuntu 6.10</td><td>Edgy Eft</td></tr>
<tr><td>Ubuntu 6.06 LTS</td><td>Dapper Drake</td></tr>
<tr><td>Ubuntu 5.10</td><td>Breezy Badger</td></tr>
<tr><td>Ubuntu 5.04</td><td>Hoary Hedgehog</td></tr>
<tr><td>Ubuntu 4.10</td><td>Warty Warthog</td></tr>
</table>
Ubuntu version numbers are also release dates: Ubuntu 14.04 was released in April 2014.
</div>
<div> </div>
<div class="col-md-6">
<h2>OSX</h2>
<table class="table-striped">
<tbody>
<tr><td>macOS 12</td><td>Monterey</td><td>2021</td></tr>
<tr><td>macOS 11</td><td>Big Sur</td><td>2020</td></tr>
<tr><td>OS X 10.15</td><td>Catalina</td><td>2019</td></tr>
<tr><td>OS X 10.14</td><td>Mojave</td><td>2018</td></tr>
<tr><td>OS X 10.13</td><td>High Sierra</td><td>2017</td></tr>
<tr><td>OS X 10.12</td><td>Sierra</td><td>2016</td></tr>
<tr><td>OS X 10.11</td><td>El Capitan</td><td>2015</td></tr>
<tr><td>OS X 10.10</td><td>Yosemite</td><td>2014</td></tr>
<tr><td>OS X 10.9</td><td>Mavericks</td><td>October 22, 2013</td></tr>
<tr><td>OS X 10.8</td><td>Mountain Lion</td><td>July 25, 2012</td></tr>
<tr><td>OS X 10.7</td><td>Lion</td><td>July 20, 2011</td></tr>
<tr><td>OS X 10.6</td><td>Snow Leopard</td><td>August 28, 2009</td></tr>
<tr><td>OS X 10.5</td><td>Leopard</td><td>October 26, 2007</td></tr>
<tr><td>OS X 10.4</td><td>Tiger</td><td>April 29, 2005</td></tr>
<tr><td>OS X 10.3</td><td>Panther</td><td>October 24, 2003</td></tr>
<tr><td>OS X 10.2</td><td>Jaguar</td><td>August 24, 2002</td></tr>
<tr><td>OS X 10.1</td><td>Puma</td><td>September 25, 2001</td></tr>
<tr><td>OS X 10.0</td><td>Cheetah</td><td>March 24, 2001</td></tr>
</tbody>
</table>
</div>
</div>Adding gphoto support for the Canon 1DS2008-08-09T16:23:00-07:002021-12-26T08:21:50-08:00shagtag:None,2008-08-09:/adding-gphoto-support-for-the-canon-1ds.html<p>Canon 1Ds HackiWiki!!</p>
<h2>2007 Aug 20: Raj and I discovered some interesting stuff!!!</h2>
<div class="highlight"><pre><span></span><code><span class="mf">606527</span> <span class="n">dest</span><span class="o">=</span><span class="n">WINDOZE</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x39</span><span class="p">,</span> <span class="kr">read</span><span class="n">_block_request</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="n">CAMERA</span><span class="p">,</span> <span class="n">offs</span><span class="o">=</span><span class="mf">0</span><span class="n">x000004da60a8</span><span class="p">,</span> <span class="kd">data</span><span class="n">_length</span><span class="o">=</span><span class="mf">0</span><span class="n">x0020</span><span class="p">,</span> <span class="n">extended_tcode</span><span class="o">=</span><span class="mf">0</span><span class="n">x0000</span><span class="p">,</span> <span class="n">ack_pending</span>
<span class="mf">606554</span> <span class="n">dest</span><span class="o">=</span><span class="n">CAMERA</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x39</span><span class="p">,</span> <span class="kr">read</span><span class="n">_block_response</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="n">WINDOZE</span><span class="p">,</span> <span class="n">complete</span><span class="p">,</span> <span class="kd">data …</span></code></pre></div><p>Canon 1Ds HackiWiki!!</p>
<h2>2007 Aug 20: Raj and I discovered some interesting stuff!!!</h2>
<div class="highlight"><pre><span></span><code><span class="mf">606527</span> <span class="n">dest</span><span class="o">=</span><span class="n">WINDOZE</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x39</span><span class="p">,</span> <span class="kr">read</span><span class="n">_block_request</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="n">CAMERA</span><span class="p">,</span> <span class="n">offs</span><span class="o">=</span><span class="mf">0</span><span class="n">x000004da60a8</span><span class="p">,</span> <span class="kd">data</span><span class="n">_length</span><span class="o">=</span><span class="mf">0</span><span class="n">x0020</span><span class="p">,</span> <span class="n">extended_tcode</span><span class="o">=</span><span class="mf">0</span><span class="n">x0000</span><span class="p">,</span> <span class="n">ack_pending</span>
<span class="mf">606554</span> <span class="n">dest</span><span class="o">=</span><span class="n">CAMERA</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x39</span><span class="p">,</span> <span class="kr">read</span><span class="n">_block_response</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="n">WINDOZE</span><span class="p">,</span> <span class="n">complete</span><span class="p">,</span> <span class="kd">data</span><span class="n">_length</span><span class="o">=</span><span class="mf">0</span><span class="n">x0020</span><span class="p">,</span> <span class="n">extended_tcode</span><span class="o">=</span><span class="mf">0</span><span class="n">x0000</span><span class="p">,</span> <span class="kd">data</span><span class="o">=</span><span class="err">[</span>
<span class="n">ffffffff</span> <span class="n">ffffffff</span> <span class="n">ffc30000</span> <span class="mf">04</span><span class="n">bf2000</span>
<span class="mf">8</span><span class="n">a98000e</span> <span class="mf">28000000</span> <span class="mf">000000</span><span class="n">ff</span> <span class="mf">00000000</span>
<span class="err">]</span><span class="p">,</span> <span class="n">ack_complete</span>
</code></pre></div>
<ul>
<li>bytes 0-3 are always(?) ffffffff</li>
<li>bytes 4-7 are always(?) ffffffff</li>
<li>bytes 8-11 are always ffc30000 -- Shag is willing to bet that this 0xffc3 is the Firewire bus address of 'WINDOZE' ...<ul>
<li>A 16-bit firewire address is divided into two parts. The top 10 bits are the bus ID and the bottom 6 are the node ID. In the case of 0xffc3, the top ten bits are 1063 (base 10), which is a special value meaning 'local bus' (<a href="http://www.linux1394.org/doc/libraw1394/intro1394.html#AEN25">reference</a>)</li>
</ul>
</li>
<li>bytes 12-15 [0x04bf2000] is the offset for the S/G list</li>
<li>byte 17 & mask 0x08: if true, then apparently means 'S/G list' - "counter packet" below byte 17 is 0x90</li>
<li>byte 19 (base 10) [0xe] above is the number of entries in the "S/G list" ...</li>
</ul>
<hr>
<div class="highlight"><pre><span></span><code><span class="mf">608909</span> <span class="n">dest</span><span class="o">=</span><span class="n">WINDOZE</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x1c</span><span class="p">,</span> <span class="kr">read</span><span class="n">_block_request</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="n">CAMERA</span><span class="p">,</span> <span class="n">offs</span><span class="o">=</span><span class="mf">0</span><span class="n">x000004bf2000</span><span class="p">,</span> <span class="kd">data</span><span class="n">_length</span><span class="o">=</span><span class="mf">0</span><span class="n">x0070</span><span class="p">,</span> <span class="n">extended_tcode</span><span class="o">=</span><span class="mf">0</span><span class="n">x0000</span><span class="p">,</span> <span class="n">ack_pending</span>
<span class="mf">608951</span> <span class="n">dest</span><span class="o">=</span><span class="n">CAMERA</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x1c</span><span class="p">,</span> <span class="kr">read</span><span class="n">_block_response</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="n">WINDOZE</span><span class="p">,</span> <span class="n">complete</span><span class="p">,</span> <span class="kd">data</span><span class="n">_length</span><span class="o">=</span><span class="mf">0</span><span class="n">x0070</span><span class="p">,</span> <span class="n">extended_tcode</span><span class="o">=</span><span class="mf">0</span><span class="n">x0000</span><span class="p">,</span> <span class="kd">data</span><span class="o">=</span><span class="err">[</span>
<span class="mf">0</span><span class="n">fb00000</span> <span class="mf">0</span><span class="n">b23f050</span> <span class="mf">10000000</span> <span class="mf">0</span><span class="n">b48d000</span>
<span class="mf">10000000</span> <span class="mf">0</span><span class="n">afe7000</span> <span class="mf">10000000</span> <span class="mf">0</span><span class="n">b468000</span>
<span class="mf">10000000</span> <span class="mf">0</span><span class="n">b449000</span> <span class="mf">20000000</span> <span class="mf">0</span><span class="n">b4ba000</span>
<span class="mf">10000000</span> <span class="mf">0</span><span class="n">a3fc000</span> <span class="mf">10000000</span> <span class="mf">0</span><span class="n">a3dd000</span>
<span class="mf">10000000</span> <span class="mf">0</span><span class="n">a9de000</span> <span class="mf">10000000</span> <span class="mf">0</span><span class="n">b437000</span>
<span class="mf">10000000</span> <span class="mf">0</span><span class="n">b488000</span> <span class="mf">10000000</span> <span class="mf">0</span><span class="n">b461000</span>
<span class="mf">20000000</span> <span class="mf">0</span><span class="n">a3ea000</span> <span class="mf">0</span><span class="n">f500000</span> <span class="mf">0</span><span class="n">b48e000</span>
<span class="err">]</span><span class="p">,</span> <span class="n">ack_complete</span>
</code></pre></div>
<p>First two bytes are length of transfer; next two bytes apparently are ignored; next four bytes is
address on WINDOZE to transfer to. After this the JPEG data starts.</p>
<hr>
<p>These wacky packets (wackets?) are from a completely different part of the file.</p>
<div class="highlight"><pre><span></span><code><span class="mf">507816</span> <span class="n">dest</span><span class="o">=</span><span class="n">CAMERA</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x39</span><span class="p">,</span> <span class="kr">read</span><span class="n">_block_response</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="n">WINDOZE</span><span class="p">,</span> <span class="n">complete</span><span class="p">,</span> <span class="kd">data</span><span class="n">_length</span><span class="o">=</span><span class="mf">0</span><span class="n">x0020</span><span class="p">,</span> <span class="n">extended_tcode</span><span class="o">=</span><span class="mf">0</span><span class="n">x0000</span><span class="p">,</span> <span class="kd">data</span><span class="o">=</span><span class="err">[</span>
<span class="n">ffffffff</span> <span class="n">ffffffff</span> <span class="n">ffc30000</span> <span class="mf">172</span><span class="n">e6380</span>
<span class="mf">8</span><span class="n">a90005c</span> <span class="mf">28000000</span> <span class="mf">00000000</span> <span class="mf">5</span><span class="n">c000000</span>
<span class="err">]</span><span class="p">,</span> <span class="n">ack_complete</span>
</code></pre></div>
<p>Above we see that byte 17 & mask 0x08 is false. So something is up. Byte 19 [0x5c] happens to represent the number of bytes in the response packet. Interesting...</p>
<div class="highlight"><pre><span></span><code><span class="mf">521267</span> <span class="n">dest</span><span class="o">=</span><span class="n">WINDOZE</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x15</span><span class="p">,</span> <span class="n">write_block_request</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="n">CAMERA</span><span class="p">,</span> <span class="n">offs</span><span class="o">=</span><span class="mf">0</span><span class="n">x0000172e6380</span><span class="p">,</span> <span class="kd">data</span><span class="n">_length</span><span class="o">=</span><span class="mf">0</span><span class="n">x005c</span><span class="p">,</span> <span class="n">extended_tcode</span><span class="o">=</span><span class="mf">0</span><span class="n">x0000</span><span class="p">,</span> <span class="kd">data</span><span class="o">=</span><span class="err">[</span>
<span class="mf">10000020</span> <span class="mf">42000000</span> <span class="mf">02000000</span> <span class="mf">1</span><span class="n">d001022</span>
<span class="mf">42000000</span> <span class="mf">7</span><span class="n">ced2501</span> <span class="mf">00000000</span> <span class="mf">2</span><span class="n">e000100</span>
<span class="mf">00010002</span> <span class="mf">00030004</span> <span class="mf">00050006</span> <span class="mf">00070008</span>
<span class="mf">0009000</span><span class="n">a</span> <span class="mf">000</span><span class="n">b000c</span> <span class="mf">000</span><span class="n">d000e</span> <span class="mf">000</span><span class="n">f0010</span>
<span class="mf">00110012</span> <span class="mf">00130014</span> <span class="mf">00150200</span> <span class="n">ff0100ff</span>
<span class="mf">0200</span><span class="n">ff01</span> <span class="mf">05</span><span class="n">ff0200</span> <span class="n">ff0106ff</span>
<span class="err">]</span><span class="p">,</span> <span class="n">ack_complete</span>
</code></pre></div>
<p>What's interesting is that the middle of the packet counts from 0x0001 to 0x0015 in 16-bit words. Then after that there appear to be six 3-byte strings: 0x0200ff, 0x0100ff, 0x0200ff, 0x0105ff, 0x0106ff.</p>
<hr>
<p>Okay I'm pretty interested in this 0x5c thing. That is a magic number that I remember from the gphoto2 Canon USB driver. 0x5c is the number of bytes in a 'release params' data structure from the camera. Release params are things like shutter speed, ISO, etc. Let's look at the gphoto2 usb source and see if anything matches up. I think this picture was captured at ISO 640... (looks at camlibs/canon source) ... let's see, for the EOS 5D, that would be a value of 0x5d, offset 0x1a bytes into the packet .... doh, doesn't match. false alarm... XXX Turns out this is because I'm looking in the wrong place! See below!</p>
<hr>
<p>Camera idle loop:</p>
<p>This sequence occurs every second under Windows. 0xffc3 is WINDOZE, 0xffc0 is CAMERA.</p>
<div class="highlight"><pre><span></span><code><span class="mf">363034</span> <span class="n">dest</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc0</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x2a</span><span class="p">,</span> <span class="n">write_quadlet_request</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc3</span><span class="p">,</span> <span class="n">offs</span><span class="o">=</span><span class="mf">0</span><span class="n">xfffff0010810</span><span class="p">,</span> <span class="kd">data</span><span class="o">=</span><span class="mf">0</span><span class="n">x0000000f</span><span class="p">,</span> <span class="n">ack_pending</span>
<span class="mf">363064</span> <span class="n">dest</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc3</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x2a</span><span class="p">,</span> <span class="n">write_response</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc0</span><span class="p">,</span> <span class="n">complete</span><span class="p">,</span> <span class="n">ack_complete</span>
<span class="mf">363090</span> <span class="n">dest</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc3</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x38</span><span class="p">,</span> <span class="kr">read</span><span class="n">_block_request</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc0</span><span class="p">,</span> <span class="n">offs</span><span class="o">=</span><span class="mf">0</span><span class="n">x000004da60d0</span><span class="p">,</span> <span class="kd">data</span><span class="n">_length</span><span class="o">=</span><span class="mf">0</span><span class="n">x0008</span><span class="p">,</span> <span class="n">extended_tcode</span><span class="o">=</span><span class="mf">0</span><span class="n">x0000</span><span class="p">,</span> <span class="n">ack_pending</span>
<span class="mf">363117</span> <span class="n">dest</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc0</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x38</span><span class="p">,</span> <span class="kr">read</span><span class="n">_block_response</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc3</span><span class="p">,</span> <span class="n">complete</span><span class="p">,</span> <span class="kd">data</span><span class="n">_length</span><span class="o">=</span><span class="mf">0</span><span class="n">x0008</span><span class="p">,</span> <span class="n">extended_tcode</span><span class="o">=</span><span class="mf">0</span><span class="n">x0000</span><span class="p">,</span> <span class="kd">data</span><span class="o">=</span><span class="err">[</span>
<span class="mf">00000000</span> <span class="mf">04</span><span class="n">da60a8</span>
<span class="err">]</span><span class="p">,</span> <span class="n">ack_complete</span>
<span class="mf">363762</span> <span class="n">dest</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc3</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x39</span><span class="p">,</span> <span class="kr">read</span><span class="n">_block_request</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc0</span><span class="p">,</span> <span class="n">offs</span><span class="o">=</span><span class="mf">0</span><span class="n">x000004da60a8</span><span class="p">,</span> <span class="kd">data</span><span class="n">_length</span><span class="o">=</span><span class="mf">0</span><span class="n">x0020</span><span class="p">,</span> <span class="n">extended_tcode</span><span class="o">=</span><span class="mf">0</span><span class="n">x0000</span><span class="p">,</span> <span class="n">ack_pending</span>
<span class="mf">363788</span> <span class="n">dest</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc0</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x39</span><span class="p">,</span> <span class="kr">read</span><span class="n">_block_response</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc3</span><span class="p">,</span> <span class="n">complete</span><span class="p">,</span> <span class="kd">data</span><span class="n">_length</span><span class="o">=</span><span class="mf">0</span><span class="n">x0020</span><span class="p">,</span> <span class="n">extended_tcode</span><span class="o">=</span><span class="mf">0</span><span class="n">x0000</span><span class="p">,</span> <span class="kd">data</span><span class="o">=</span><span class="err">[</span>
<span class="n">ffffffff</span> <span class="n">ffffffff</span> <span class="n">ffc30000</span> <span class="mf">0</span><span class="n">cf5ed20</span>
<span class="mf">8</span><span class="n">a900054</span> <span class="mf">03000000</span> <span class="mf">54000000</span> <span class="mf">00000000</span>
<span class="err">]</span><span class="p">,</span> <span class="n">ack_complete</span>
<span class="mf">364702</span> <span class="n">dest</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc3</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x16</span><span class="p">,</span> <span class="n">write_block_request</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc0</span><span class="p">,</span> <span class="n">offs</span><span class="o">=</span><span class="mf">0</span><span class="n">x00000cf5ed20</span><span class="p">,</span> <span class="kd">data</span><span class="n">_length</span><span class="o">=</span><span class="mf">0</span><span class="n">x0054</span><span class="p">,</span> <span class="n">extended_tcode</span><span class="o">=</span><span class="mf">0</span><span class="n">x0000</span><span class="p">,</span> <span class="kd">data</span><span class="o">=</span><span class="err">[</span>
<span class="mf">70000900</span> <span class="mf">00000058</span> <span class="mf">00000000</span> <span class="mf">80000000</span>
<span class="mf">00000000</span> <span class="mf">00000000</span> <span class="mf">00000000</span> <span class="mf">00000000</span>
<span class="mf">00000000</span> <span class="mf">00000000</span> <span class="mf">00000000</span> <span class="mf">00000000</span>
<span class="mf">00000000</span> <span class="mf">00000000</span> <span class="mf">00000000</span> <span class="mf">00000000</span>
<span class="mf">00000000</span> <span class="mf">00000000</span> <span class="mf">00000000</span> <span class="mf">00000000</span>
<span class="mf">00000000</span>
<span class="err">]</span><span class="p">,</span> <span class="n">ack_complete</span>
<span class="mf">365127</span> <span class="n">dest</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc3</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x0b</span><span class="p">,</span> <span class="n">write_block_request</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc0</span><span class="p">,</span> <span class="n">offs</span><span class="o">=</span><span class="mf">0</span><span class="n">x000900000020</span><span class="p">,</span> <span class="kd">data</span><span class="n">_length</span><span class="o">=</span><span class="mf">0</span><span class="n">x0008</span><span class="p">,</span> <span class="n">extended_tcode</span><span class="o">=</span><span class="mf">0</span><span class="n">x0000</span><span class="p">,</span> <span class="kd">data</span><span class="o">=</span><span class="err">[</span>
<span class="mf">41000000</span> <span class="mf">04</span><span class="n">da60a8</span>
<span class="err">]</span><span class="p">,</span> <span class="n">ack_complete</span>
</code></pre></div>
<hr>
<p><strong>Aha! Byte diff for an F-stop setting change</strong></p>
<div class="highlight"><pre><span></span><code><span class="mf">325342</span> <span class="n">dest</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc0</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x09</span><span class="p">,</span> <span class="n">write_quadlet_request</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc3</span><span class="p">,</span> <span class="n">offs</span><span class="o">=</span><span class="mf">0</span><span class="n">xfffff0010810</span><span class="p">,</span> <span class="kd">data</span><span class="o">=</span><span class="mf">0</span><span class="n">x0000000f</span><span class="p">,</span> <span class="n">ack_pending</span>
<span class="mf">325368</span> <span class="n">dest</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc3</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x09</span><span class="p">,</span> <span class="n">write_response</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc0</span><span class="p">,</span> <span class="n">complete</span><span class="p">,</span> <span class="n">ack_complete</span>
<span class="mf">325393</span> <span class="n">dest</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc3</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x38</span><span class="p">,</span> <span class="kr">read</span><span class="n">_block_request</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc0</span><span class="p">,</span> <span class="n">offs</span><span class="o">=</span><span class="mf">0</span><span class="n">x000004da60a8</span><span class="p">,</span> <span class="kd">data</span><span class="n">_length</span><span class="o">=</span><span class="mf">0</span><span class="n">x0008</span><span class="p">,</span> <span class="n">extended_tcode</span><span class="o">=</span><span class="mf">0</span><span class="n">x0000</span><span class="p">,</span> <span class="n">ack_pending</span>
<span class="mf">325419</span> <span class="n">dest</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc0</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x38</span><span class="p">,</span> <span class="kr">read</span><span class="n">_block_response</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc3</span><span class="p">,</span> <span class="n">complete</span><span class="p">,</span> <span class="kd">data</span><span class="n">_length</span><span class="o">=</span><span class="mf">0</span><span class="n">x0008</span><span class="p">,</span> <span class="n">extended_tcode</span><span class="o">=</span><span class="mf">0</span><span class="n">x0000</span><span class="p">,</span> <span class="kd">data</span><span class="o">=</span><span class="err">[</span>
<span class="mf">00000000</span> <span class="mf">04</span><span class="n">da60d0</span>
<span class="err">]</span><span class="p">,</span> <span class="n">ack_complete</span>
<span class="mf">325751</span> <span class="n">dest</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc3</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x39</span><span class="p">,</span> <span class="kr">read</span><span class="n">_block_request</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc0</span><span class="p">,</span> <span class="n">offs</span><span class="o">=</span><span class="mf">0</span><span class="n">x000004da60d0</span><span class="p">,</span> <span class="kd">data</span><span class="n">_length</span><span class="o">=</span><span class="mf">0</span><span class="n">x0020</span><span class="p">,</span> <span class="n">extended_tcode</span><span class="o">=</span><span class="mf">0</span><span class="n">x0000</span><span class="p">,</span> <span class="n">ack_pending</span>
<span class="mf">325780</span> <span class="n">dest</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc0</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x39</span><span class="p">,</span> <span class="kr">read</span><span class="n">_block_response</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc3</span><span class="p">,</span> <span class="n">complete</span><span class="p">,</span> <span class="kd">data</span><span class="n">_length</span><span class="o">=</span><span class="mf">0</span><span class="n">x0020</span><span class="p">,</span> <span class="n">extended_tcode</span><span class="o">=</span><span class="mf">0</span><span class="n">x0000</span><span class="p">,</span> <span class="kd">data</span><span class="o">=</span><span class="err">[</span>
<span class="n">ffffffff</span> <span class="n">ffffffff</span> <span class="n">ffc30000</span> <span class="mf">0775</span><span class="n">f480</span>
<span class="mf">8</span><span class="n">a900054</span> <span class="mf">28000000</span> <span class="mf">00000000</span> <span class="mf">54000000</span>
<span class="err">]</span><span class="p">,</span> <span class="n">ack_complete</span>
<span class="mf">343041</span> <span class="n">dest</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc3</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x15</span><span class="p">,</span> <span class="n">write_block_request</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc0</span><span class="p">,</span> <span class="n">offs</span><span class="o">=</span><span class="mf">0</span><span class="n">x00000775f480</span><span class="p">,</span> <span class="kd">data</span><span class="n">_length</span><span class="o">=</span><span class="mf">0</span><span class="n">x0054</span><span class="p">,</span> <span class="n">extended_tcode</span><span class="o">=</span><span class="mf">0</span><span class="n">x0000</span><span class="p">,</span> <span class="kd">data</span><span class="o">=</span><span class="err">[</span>
<span class="mf">10000020</span> <span class="mf">4</span><span class="n">c000000</span> <span class="mf">02000000</span> <span class="mf">25001022</span>
<span class="mf">4</span><span class="n">c000000</span> <span class="mf">38</span><span class="n">e92501</span> <span class="mf">00000000</span> <span class="mf">0</span><span class="n">a000000</span>
<span class="mf">30000000</span> <span class="mf">20</span><span class="n">ff0100</span> <span class="mf">0000</span><span class="n">ffff</span> <span class="mf">04010000</span>
<span class="mf">03020000</span> <span class="mf">09030000</span> <span class="mf">007</span><span class="n">f7f7f</span> <span class="mf">70005800</span>
<span class="mf">2</span><span class="n">d007800</span> <span class="mf">0000</span><span class="n">ff00</span> <span class="mf">20005800</span> <span class="mf">64006400</span>
<span class="mf">64000100</span>
<span class="err">]</span><span class="p">,</span> <span class="n">ack_complete</span>
</code></pre></div>
<p>Take a look at byte 64 [0x2d] in the above packet. That was with one particular aperture setting -- not sure if it's 4.5 or 5.0. Diffing this against traffic from changing the wheel back changed byte 64 to 0x2b. Woo hoo!</p>
<hr>
<p>One other thing from that last aperture wheel setting change. How did the computer know that the camera settings had changed, and that it should query them? Take a look at the write_block_request in the idle loop packets above. Then check this out (scroll down to the write_block_request pkt):</p>
<div class="highlight"><pre><span></span><code><span class="mf">289212</span> <span class="n">dest</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc0</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x05</span><span class="p">,</span> <span class="n">write_quadlet_request</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc3</span><span class="p">,</span> <span class="n">offs</span><span class="o">=</span><span class="mf">0</span><span class="n">xfffff0010810</span><span class="p">,</span> <span class="kd">data</span><span class="o">=</span><span class="mf">0</span><span class="n">x0000000f</span><span class="p">,</span> <span class="n">ack_pending</span>
<span class="mf">289242</span> <span class="n">dest</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc3</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x05</span><span class="p">,</span> <span class="n">write_response</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc0</span><span class="p">,</span> <span class="n">complete</span><span class="p">,</span> <span class="n">ack_complete</span>
<span class="mf">289269</span> <span class="n">dest</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc3</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x38</span><span class="p">,</span> <span class="kr">read</span><span class="n">_block_request</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc0</span><span class="p">,</span> <span class="n">offs</span><span class="o">=</span><span class="mf">0</span><span class="n">x000004da60a8</span><span class="p">,</span> <span class="kd">data</span><span class="n">_length</span><span class="o">=</span><span class="mf">0</span><span class="n">x0008</span><span class="p">,</span> <span class="n">extended_tcode</span><span class="o">=</span><span class="mf">0</span><span class="n">x0000</span><span class="p">,</span> <span class="n">ack_pending</span>
<span class="mf">289296</span> <span class="n">dest</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc0</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x38</span><span class="p">,</span> <span class="kr">read</span><span class="n">_block_response</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc3</span><span class="p">,</span> <span class="n">complete</span><span class="p">,</span> <span class="kd">data</span><span class="n">_length</span><span class="o">=</span><span class="mf">0</span><span class="n">x0008</span><span class="p">,</span> <span class="n">extended_tcode</span><span class="o">=</span><span class="mf">0</span><span class="n">x0000</span><span class="p">,</span> <span class="kd">data</span><span class="o">=</span><span class="err">[</span>
<span class="mf">00000000</span> <span class="mf">04</span><span class="n">da60d0</span>
<span class="err">]</span><span class="p">,</span> <span class="n">ack_complete</span>
<span class="mf">289881</span> <span class="n">dest</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc3</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x39</span><span class="p">,</span> <span class="kr">read</span><span class="n">_block_request</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc0</span><span class="p">,</span> <span class="n">offs</span><span class="o">=</span><span class="mf">0</span><span class="n">x000004da60d0</span><span class="p">,</span> <span class="kd">data</span><span class="n">_length</span><span class="o">=</span><span class="mf">0</span><span class="n">x0020</span><span class="p">,</span> <span class="n">extended_tcode</span><span class="o">=</span><span class="mf">0</span><span class="n">x0000</span><span class="p">,</span> <span class="n">ack_pending</span>
<span class="mf">289908</span> <span class="n">dest</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc0</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x39</span><span class="p">,</span> <span class="kr">read</span><span class="n">_block_response</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc3</span><span class="p">,</span> <span class="n">complete</span><span class="p">,</span> <span class="kd">data</span><span class="n">_length</span><span class="o">=</span><span class="mf">0</span><span class="n">x0020</span><span class="p">,</span> <span class="n">extended_tcode</span><span class="o">=</span><span class="mf">0</span><span class="n">x0000</span><span class="p">,</span> <span class="kd">data</span><span class="o">=</span><span class="err">[</span>
<span class="n">ffffffff</span> <span class="n">ffffffff</span> <span class="n">ffc30000</span> <span class="mf">0</span><span class="n">cf5ed20</span>
<span class="mf">8</span><span class="n">a900054</span> <span class="mf">03000000</span> <span class="mf">54000000</span> <span class="mf">00000000</span>
<span class="err">]</span><span class="p">,</span> <span class="n">ack_complete</span>
<span class="mf">290452</span> <span class="n">dest</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc3</span><span class="p">,</span> <span class="n">tl</span><span class="o">=</span><span class="mf">0</span><span class="n">x11</span><span class="p">,</span> <span class="n">write_block_request</span><span class="p">,</span> <span class="n">src</span><span class="o">=</span><span class="mf">0</span><span class="n">xffc0</span><span class="p">,</span> <span class="n">offs</span><span class="o">=</span><span class="mf">0</span><span class="n">x00000cf5ed20</span><span class="p">,</span> <span class="kd">data</span><span class="n">_length</span><span class="o">=</span><span class="mf">0</span><span class="n">x0054</span><span class="p">,</span> <span class="n">extended_tcode</span><span class="o">=</span><span class="mf">0</span><span class="n">x0000</span><span class="p">,</span> <span class="kd">data</span><span class="o">=</span><span class="err">[</span>
<span class="mf">70000900</span> <span class="mf">00000058</span> <span class="mf">00000000</span> <span class="mf">80000000</span>
<span class="mf">00000000</span> <span class="mf">02000000</span> <span class="mf">0</span><span class="n">a000000</span> <span class="mf">10000000</span>
<span class="mf">1</span><span class="n">b000000</span> <span class="mf">00000000</span> <span class="mf">00000000</span> <span class="mf">00845400</span>
<span class="n">f4ff9500</span> <span class="n">d17b4100</span> <span class="mf">0</span><span class="n">f05ffff</span> <span class="n">ffff0106</span>
<span class="n">ffffffff</span> <span class="mf">0102</span><span class="n">ffff</span> <span class="n">ffff0200</span> <span class="n">ff0100ff</span>
<span class="mf">0200</span><span class="n">ff01</span>
<span class="err">]</span><span class="p">,</span> <span class="n">ack_complete</span>
</code></pre></div>
<p>See all that junk at the end? I think that's how the computer knows to follow up queries.</p>
<hr>
<p>Okay, just changed ISO and did the same thing as the aperture thing. Changed ISO 250 to 320. Byte 66 changed to 0x7b. Changed back. Byte 66 changed to 0x78. Those two values match exactly with the shutter speed table in gphoto2 camlibs/canon. Shutter speed offset is 0x1e, and aperture offset is 0x1c in the gphoto USB code; this seems to jive with those defines if we consider the 'start' of the release params payload to start at byte 36 of the packet.</p>
<p>It's interesting that the capture.log.txt that we took has 0xff for both of these parameters. Perhaps that is because we took those shots in full-auto mode?</p>
<p>Anyway, it looks like the magic hex string to search for to find get-release-param commands is '8a900054 28'.</p>
<hr>
<p><strong>libraw1394</strong></p>
<p>I'll bet this is what we'd need to use to start testing these commands out: <a href="http://www.linux1394.org/doc/libraw1394/">http://www.linux1394.org/doc/libraw1394/</a></p>
<p>XXX On the other hand, if this device really just speaks SCSI, then we don't need libraw1394 at all..</p>
<hr>
<h2>Arrrr!</h2>
<p><img alt="paul_and_raj_r0xx0rs" src="/images/paul_and_raj_r0xx0rs.jpg"></p>
<hr>
<h2>2007 Aug 24: SBP and SCSI and Wireshark oh my</h2>
<p><strong>SCSI and SBP2</strong></p>
<p>So today Raj and shag had an interesting thought. We noticed that when we plugged in the camera to our Linux box that <a href="http://t10.org/ftp/t10/drafts/sbp2/sbp2r04.pdf">SBP-2</a> kicked in and the SCSI layer identified the camera and decided it was a scanner (at least in terms of SCSI). This made us think, "Selves, what if we nosy'd the Linux->camera communication upon initial plugin and compared that the communication on the Windows side to see if that was using SBP-2 also?" So we did. And guess what? We found a lot of similar packets! So now we think that the Canon software is just sending down SCSI commands over SBP-2 over Firewire to get the camera to do things.</p>
<p>So then the question became, well, how do we snoop SCSI? And it turns out that we could not find any SCSI snoopers that were not hardware based and therefore cost a gazillion dollars and probably did not have any Firewire ports on them anyway. ... at least until we discovered that Wireshark nee Ethereal had an interesting thingamabob called packet-scsi.c written by some dude at Cicsoc who probably got paid to write that thing for iSCSI. but maybe we can use that too!!! all wee need to do is to write something where Wireshark can load up nosy-dump files, then we need to write a dissector thingie for SBP-2 (since that does not appear to exist). Then in the best of all possible worlds, packet-scsi would then magically parse all of the data.</p>
<p>yeah.</p>
<hr>
<p>So we think we'll also probably have to write a basic Firewire -- really, a nosy-dump file format -- dissector. The traces we've read look pretty simple so far in terms of commands - there aren't any isochronous transfers, all we should need are:</p>
<ul>
<li>self id</li>
<li>bus reset</li>
<li>{read,write}<em _response=",response">{quadlet,block}</em></li>
</ul>
<p>looks like we can pull the format right out of the nosy-dump.c source code, <a href="http://bitplanet.net/">krh's</a> structs are really nicely formatted and this should be straightforward.</p>
<ul>
<li>Wireshark Developer's Guide: <a href="http://www.wireshark.org/download/docs/developer-guide-us.pdf">PDF</a></li>
</ul>
<hr>
<p>So to write a new file format importer for Wireshark, one must modify this library that comes with it called Wiretap. unfortunately the current format of nosy-dump files doesn't have any magic numbers that can be used to identify them, so probably the best thing to do is to prepend some magic number/string with an external tool, and modify nosy-dump to prepend these itself for future revs.</p>
<h2>2007 Nov 1: nosy-dump file format</h2>
<p>So the current rev of nosy-dump has a really simple file format: PASCAL strings (basically). It writes exactly what nosy-dump receives from the kernel: an int, representing the number of bytes to follow, and then writes the bytes.</p>
<p><strong>Hack libpcap instead</strong></p>
<p>Probably the best thing to do is to hack FireWire sniffing support into libpcap. libpcap already has Bluetooth and USB sniffing support in it...</p>
<h2>2007 Late November: adding FireWire support to Wireshark</h2>
<p>Shelved the libpcap idea for the time being. Wrote a basic nosydump wiretap reader, and hacked up basic Firewire phy & link protocol dissectors to Wireshark. Next step is to hack up something for SBP2.</p>
<p><img alt="wireshark" src="/images/wireshark1.jpg"></p>Adding gphoto support for the Canon 5D2008-08-09T16:23:00-07:002021-12-26T08:21:50-08:00rajtag:None,2008-08-09:/adding-gphoto-support-for-the-canon-5d.html<h3>gphoto for Canon EOS 5D compilation instructions</h3>
<h4>you MUST use svn rev 10595 or greater. We tested with 10597</h4>
<p>Building gphoto on Ubuntu 6.10 (note that default 2.6.17 kernel spinlocks, use custom kernel)</p>
<div class="highlight"><pre><span></span><code><span class="c1">#check out trunk. The checkout will fail when trying to pull the libusb directory …</span></code></pre></div><h3>gphoto for Canon EOS 5D compilation instructions</h3>
<h4>you MUST use svn rev 10595 or greater. We tested with 10597</h4>
<p>Building gphoto on Ubuntu 6.10 (note that default 2.6.17 kernel spinlocks, use custom kernel)</p>
<div class="highlight"><pre><span></span><code><span class="c1">#check out trunk. The checkout will fail when trying to pull the libusb directory</span>
svn co https://gphoto.svn.sourceforge.net/svnroot/gphoto/trunk gphoto
<span class="nb">cd</span> gphoto
<span class="c1">#fix broken checkout:</span>
svn up bindings/ gphoto2/ gphoto2-manual/ gphotofs/ gphoto-import/ gphoto-suite/ gtkam/ libgphoto2/ m4/ playground/ project-3/ website/
<span class="c1">#build libgphoto2</span>
<span class="nb">cd</span> libgphoto2
autoreconf -is
./configure --with-camlibs<span class="o">=</span>canon --prefix<span class="o">=</span>/scribe
make
sudo make install
<span class="c1">#build gphoto2</span>
<span class="nb">cd</span> ../gphoto2
autoreconf -is
./configure --with-libgphoto2<span class="o">=</span>/scribe --prefix<span class="o">=</span>/scribe
make
sudo make install
</code></pre></div>
<h3>linux kernel issues</h3>
<p>The kernel used by default in Ubuntu 7.04 (feisty) contains usb autosuspend code. Unfortunately, the Canon EOS 5D does not wake back up properly, which makes capturing images impossible. <a href="http://lkml.org/lkml/2007/8/16/330">A patch has been submitted</a>, but until it is accepted, you will have to compile a kernel with USB autosuspend turned off. <strong>Update</strong>: patch <a href="http://sourceforge.net/mailarchive/message.php?msg_id=20070822220805.GB30603%40kroah.com">accepted</a>, should be in 2.6.23 kernel. Instructions on how we compiled the kernel are below.</p>
<p>The kernel used by default in Ubuntu 6.10 (edgy) contains a bug that causes a spinlock in the jbd layer. This bug will cause a machine to hang in a completely unresponsive state, although you can use a serial console to see assertions firing in jbd. One workaround is to use a newer version of Ubuntu (7.04) with autosuspend turned off. Another is to use ext2 intead of ext3.</p>
<h4>Compiling a custom linux kernel</h4>
<p>We copied the .config file for the 2.6.17 kernel used by Ubuntu and slightly modified it to turn off usb autosuspend and include the nvidia sata driver.</p>
<div class="highlight"><pre><span></span><code><span class="c1"># Install kernel build pre-reqs</span>
sudo apt-get install linux-kernel-devel initrd-tools libncurses-dev
<span class="nb">cd</span> ~
mkdir linux
<span class="nb">cd</span> linux
<span class="c1"># Download stable kernel tree from www.kernel.org ('F' link)</span>
wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.22.2.tar.bz2
tar xjvf linux-2.6.22.2.tar.bz2
<span class="nb">cd</span> linux-2.6.22.2
<span class="c1"># Copy the Ubuntu kernel .config file into your source tree:</span>
cp /usr/src/linux-headers-2.6.17-12-generic/.config .
<span class="c1"># Update the .config with any new entries between 2.6.17 and the current kernel</span>
rev
<span class="c1"># (You just want to wail on the ENTER key here to select all of the defaults)</span>
yes <span class="s1">''</span> <span class="p">|</span> make oldconfig
<span class="c1"># Edit the .config to disable CONFIG_USB_SUSPEND</span>
perl -p -i.orig -e <span class="s1">'s/CONFIG_USB_SUSPEND=y/CONFIG_USB_SUSPEND=n/'</span> .config
<span class="c1"># Enable the Nvidia SATA driver</span>
<span class="nb">echo</span> <span class="nv">CONFIG_ATA</span><span class="o">=</span>m >> .config
<span class="nb">echo</span> <span class="nv">CONFIG_ATA_ACPI</span><span class="o">=</span>y >> .config
<span class="nb">echo</span> <span class="nv">CONFIG_SATA_NV</span><span class="o">=</span>m >> .config
<span class="c1"># Build kernel (dual-cpu system)</span>
make -j3 all
<span class="c1"># Install modules & kernel</span>
sudo make install modules_install
<span class="c1"># Build the boot-time ramdisk of loadable modules, etc.</span>
sudo mkinitrd -o /boot/initrd.img-2.6.22.2 <span class="m">2</span>.6.22.2
<span class="c1"># Add grub boot section</span>
<span class="nb">echo</span> <span class="s1">'Now edit /boot/grub/menu.lst to add a section for the new kernel'</span>
</code></pre></div>
<h4>Install custom linux kernel and new nvidia drivers</h4>
<p>On homeserver:</p>
<div class="highlight"><pre><span></span><code><span class="nb">cd</span> ~pjw
sudo ssh root@host <span class="s1">'cd /;tar -xjvf -'</span> < new-kernel.tar.bz2
sudo ssh root@host <span class="s1">'cd /home/scribe;tar -xzvf -'</span> < new-kernel-source.tar.gz
scp NVIDIA-Linux-x86-100.14.11-pkg1.run scribe@host:
</code></pre></div>
<p>Now on the scribe node:</p>
<div class="highlight"><pre><span></span><code>sudo /etc/init.d/gdm stop
sudo sh NVIDIA-Linux-x86-100.14.11-pkg1.run
</code></pre></div>