Here's how I got it to sort of work.
1. Install from "Setup.exe" on the CD.
2. Make sure it's installing to a "win98" bottle.
3. Go through the program's normal installation, including DirectX 3.
4. Click through all the "Can't GetProcAddress() for WINNLS32.DLL WINNLS32EnableIME()" errors that pop up during install. You might have to alt-tab to get to them sometimes.
After installation:
1. Go to the Control Panel for the bottle you installed to.
2. Make a launcher for the program by clicking "Run...", browsing to "fake_c_drive/SEGA/Sonic3K/RUNSONIC.EXE", and clicking "Create Shortcut".
3. After launching the program, go to Options, disable background music, and click "Check this box if you experience graphical problems".
4. Install SEGA's fast CPU fix at http://www2.sega.com/support/pcdownloads/sonic_knuckles_patch/s3kpatch.exe
Limitations & problems:
1. There's no music.
2. There is no window border.
3. Fullscreen mode doesn't work.
4. Sometimes the game randomly runs really fast again for a split second.