Monday, March 14, 2005

Beat the clock

The aforementioned clock problem was noticed early on while running 2.6.11 x86_64 kernels on this new machine. The clock ran 3x faster than it should have. So a minute only took 20 seconds in earth time. I searched all over and found several other people having similar issues, all of them with some type of Athlon. Usually the 3000+ just like mine. Most of them had laptop systems. I'm using a microATX board, the MSI RS480M2-IL. It will take a while to get to the solution, but I did find one.

Nobody had any solutions posted. I started out trying some different bios settings. One of these settings gave me a no-boot situation where the machine stopped mid-POST (after CPU, but before memory tests). It was the A20 setting which had options of Fast or Off. I didn't know what A20 was, but I thought, "hey, if it's fast now, maybe turning it off would slow the clock down."

So now the machine doesn't boot.
Step 1: clear the CMOS. I tried using the jumper for this purpose. No dice. Took the battery out and unplugged power. Nope.
Step 2: Remove stuff... pci cards, memory, disks. I even reseated the CPU. Nope
Step 3: What a pain... I'm actually considering the possibility that I need a new BIOS or a new board. I investigate the BIOS route, including how to build an EEPROM flasher. Or buying a new BIOS online. I bet someone at work somewhere has an EEPROM flasher, afterall, I do work in Silicon Valley now.
Step 4: Try the BIOS recovery procedure from MSI's tech support site. This requires finding a floppy drive and even more rare, a floppy disk. I disect one of my Dell's for the floppy drive, and wipe one of the driver disks that came with the mobo. After all that, the BIOS recovery doesn't even kick off like it was supposed to.
Step 5: It stops at the memory test... maybe if I try completely new memory. I pull the memory out of my Dell, and as I am putting it in, I realize the light on the optical mouse I've been using is still lit up.

Eureka! All the power is disconnected from the machine, however it's connected it to a KVM, which is also connected to another Dell which is powered up. So I unplug it from the KVM, pull out the battery, and wallah! the next boot worked. So that CMOS clearing jumper either doesn't work at all, or doesn't work when there's a powersource other than the battery on the motherboard.

Ok, after alllllll that.... I'm back to a bootable machine which runs 3x faster by the clock.

I tried booting an i386 kernel from the Fedora cd, and the clock speed was normal there, so I start looking into the kernel timer routines. I spent a few hours studying the code and undestanding more of it than I cared to. Nothing obvious stuck out. I wondered if maybe the timesource was different than what they expected. I learned all kinds of stuff about 8253 PIT's and HPET's and stuff no Java programmer should ever care about. It was fun.

I noticed how similar the x86_64 and i386 timer code was, but there were some differences. It was obvious the 64bit code had been forked from the 386 code some time back. So I started looking around on the linux kernel mailing list. I stumbled upon this patch by John Stultz, which he just happened to post a new version of hours before I started this kernel investigation: http://lkml.org/lkml/2005/3/11/309

I started trying to apply John's patches. They required some minor tweaking to get it to compile, but in the end I had a kernel rpm with the patch, and booted it up and
WoooHOOOO! A minute takes a minute now. Pretty important thing for a video recorder that schedules recording tv shows that come on at certain times. :-)

Sunday, March 13, 2005

Step 2: Install linux

This has by far, been the most frustrating step so far, and I will probably make more than one post about it cause it's not done yet. Many issues have cropped up because of my hardware choices.

Here's what I've done so far:

  1. Downloaded Fedora Core 3 x86_64 cd iso's.
  2. The installer couldn't see my SATA drive. I tried loading different drivers. I started with the Silicon Image driver because I knew that's what the motherboard used from various sources online and the driver disk included with the mobo.
  3. None of the drivers worked, so I tried changing the setting in BIOS to use the SATA RAID feature instead of IDE emulation. That didn't work either.
  4. Punt: installed an old IDE disk from my old machine that was dead figuring I could install a newer kernel version (FC3 ships with 2.6.9)
  5. Installed on the old IDE drive. This was a major PITA too. The install was taking forever. It seemed like it was having trouble reading the cd's. I switched into the other virtual terminals on the installer with ctrl-alt-f2, f3, etc. to read the logs. There were errors there about timeouts while reading from hda (the dvd/cd drive).
  6. Since the DVD player was brand new and a cheapo OEM version, I thought maybe it was bad or the cable was bad. I tried replacing each with known working cables/players from other machines. Still no luck.
  7. So I downloaded and burned the dvd iso instead of the cd iso's and just let the install run until it completed. This took half a dozen hours.
  8. Finally, the machine is running linux. But it's really really slow, just like the install was, and my SATA drive is still not visible. So I searched and found that the SATA driver was updated by someone from ATI in kernel 2.6.11. So I installed the 2.6.11 rpm from Fedora rawhide.
  9. Upon reboot, the SATA drive was visible, and the machine ran much much faster. In fact, the clock ran at 3x the normal speed. Like it gained 1 minute every 20 seconds. Even NTP wouldn't be able to correct for that amount of drift. This 3x clock problem deserves it's own post.
  10. I wanted to try installing only on the SATA drive cause the goal is to not have the old drives in this machine (they're noisy, and I don't want a noisy machine in my living room). So I rsynced all the Fedora FC4 rawhide rpms and built an iso dvd.
  11. Installed on the SATA drive from the FC4 iso. However it couldn't boot from the SATA drive. So I have to use the boot partition on the old IDE drive.
Linux is installed and running, but the 3x problem has to be fixed. MythTV's recording schedule will be way out of whack if I can't fix it. And I still have a noisy old disk in there.

My linux pvr project: step 1, build the box

I was getting bored at home (new in town, not too many friends here to hang out with yet), so I decided to build a new linux personal video recorder machine. It's turned out to be a fun and challenging project. More challenging than I expected.

I picked out the components at MicroCenter in Santa Clara last week. Here's what I got:

Motherboard: MSI RS480M2-IL
I picked this one cause it was inexpensive and had onboard video including tv-out and svideo. It's a socket 939 AMD board with SATA and all kinds of other good stuff. More on this later when I discuss getting linux installed and working.

CPU: AMD Athlon 3000+ socket 939

PVR card: Hauppage WinTV PVR-250

HDD: Maxtor 300GB SATA 7200RPM

Case: Antec Sonata

RAM: 512MB

All together, the components cost about $1000. Pretty good for a 64 bit socket 939 box (128-bit memory access path instead of 64 in socket 754)

It took about an hour to assemble the box. The hardest part there were getting the front panel usb, firewire, and sound connectors hooked up to the right jumpers on the motherboard.

Up next: installing linux