Caron WillsBut the report did not work. My results did not show up and I was quickly frustrated. This is where I considered giving up.
I don't need to have a beer, right?
I mean, that's really a simple carrot and nothing else. I wanted to submit the results but if it won't run, it won't run; and it did run but for some reason it didn't report.This is where the community comes in. This is where we often don't realize how many people are around us willing to help. I dropped a hint at my frustrations into chat and rather than getting the quick response of "that's too bad", I got a question:
How close are you?
And I don't know. And I have to be walked to the log file that wt-daily has very helpfully left in my folders. The location is:
~/wt-daily/winetest/wine/
The log file is called:
wt-bot-(yourtag).log
It's lines and lines of useful information about all the tests that ran. The first thing I learn is that WineTest really did run and it really did submit results to WineHQ. It took it almost 30 minutes to run... and it had 54 failures.
I report the number of failures and I get another question:
How many skips?
I confess I don't know. I didn't skip anything.
Now I get the explanation:
Wine's Conformance Tests are set to accept and publish results with 50 or less failures and 10 or less skips. Both requirements must be met to be featured on the results page. It sounds like you didn't skip anything, add a line to your configuration file and skip a test or two that fails and you will get it.
I'm skeptical because I've already put a bit of effort into getting this to run and I've watched a few other people grumble about libraries and dependencies but there is encouragement in that explanation: you will get it.
I started at the bottom of the logfile and looked for the failing tests. They were fairly easy to identify and so I added some tests to skip to my configuration file.
Look for lines like this:
Running tests - 53 failures Running: windowscodecs:tiffformat (541 of 579) 2015-10-05 11:28:16.135 wine[44040:808186] void *__CFPasteboardAllocatorAllocate(CFIndex) : Lock timeout Assertion failed: (sizeof(uint64)==8), function TIFFClientOpen, file tif_open.c, line 99. wine: Assertion failed at address 0x000b:0x9b5a062a (thread 0063), starting debugger... 2015-10-05 11:28:16.139 wine[44040:808186] void *__CFPasteboardAllocatorAllocate(CFIndex) : Lock timeout 2015-10-05 11:28:16.141 wine[44040:808186] void *__CFPasteboardAllocatorAllocate(CFIndex) : Lock timeout 2015-10-05 11:28:16.144 wine[44040:808186] void *__CFPasteboardAllocatorAllocate(CFIndex) : Lock timeout 2015-10-05 11:28:16.147 wine[44040:808186] void *__CFPasteboardAllocatorAllocate(CFIndex) : Lock timeout 2015-10-05 11:28:16.149 wine[44040:808186] void *__CFPasteboardAllocatorAllocate(CFIndex) : Lock timeout 2015-10-05 11:28:16.151 wine[44040:808186] void *__CFPasteboardAllocatorAllocate(CFIndex) : Lock timeout 2015-10-05 11:28:16.154 wine[44040:808186] void *__CFPasteboardAllocatorAllocate(CFIndex) : Lock timeout 2015-10-05 11:28:16.156 wine[44040:808186] void *__CFPasteboardAllocatorAllocate(CFIndex) : Lock timeout 2015-10-05 11:28:16.158 wine[44040:808186] void *__CFPasteboardAllocatorAllocate(CFIndex) : Lock timeout 2015-10-05 11:28:16.160 wine[44040:808186] void *__CFPasteboardAllocatorAllocate(CFIndex) : Lock timeout 2015-10-05 11:28:16.163 wine[44040:808186] void *__CFPasteboardAllocatorAllocate(CFIndex) : Lock timeout 2015-10-05 11:28:16.165 wine[44040:808186] void *__CFPasteboardAllocatorAllocate(CFIndex) : Lock timeout 2015-10-05 11:28:16.168 wine[44040:808186] void *__CFPasteboardAllocatorAllocate(CFIndex) : Lock timeout 2015-10-05 11:28:16.170 wine[44040:808186] void *__CFPasteboardAllocatorAllocate(CFIndex) : Lock timeout 2015-10-05 11:28:16.173 wine[44040:808186] void *__CFPasteboardAllocatorAllocate(CFIndex) : Lock timeout 2015-10-05 11:28:16.175 wine[44040:808186] void *__CFPasteboardAllocatorAllocate(CFIndex) : Lock timeout 2015-10-05 11:28:16.177 wine[44040:808186] void *__CFPasteboardAllocatorAllocate(CFIndex) : Lock timeout 2015-10-05 11:28:16.180 wine[44040:808186] void *__CFPasteboardAllocatorAllocate(CFIndex) : Lock timeout 2015-10-05 11:28:16.182 wine[44040:808186] void *__CFPasteboardAllocatorAllocate(CFIndex) : Lock timeout Running tests - 54 failures
Look for a line where the test failures increases. The amount of failures before the test is 53 and after windowscodecs:tiffformat runs, there are 54.
This is where the "windowscodecs:tiffformat" test failed. Adding a skip to that test looks like this in wt-daily's configuration file:
email="your email address here" tagmac="a-tag" descmac="A description of your system for Wine developers" excludemac="windowscodecs:tiffformat"
I need more than one test skipped to get to that 50 or less mark so my configuration file ends up looking like this:
email="caron@codeweavers.com" tagmac="CWtestbox004" descmac="OS X 10.11 dual GPU AMD Radeon R9 M370X and Intel Iris Pro" excludemac="windowscodecs:tiffformat windowscodecs:converter user32:msg gdi32:dc dwrite:font"
And it worked:
This is cause for great celebration. Not only is WineTest running on my system, it's reporting on WineTest. I made several Macs run and after a few reports came online, those watching the results come in realized that the name "CWtestbox###" was really horrible for identifying which box was which at a glance. We needed to summarize the box in 20 characters or less and still be able to tell who was who among our Macs.
This is the information we wanted to know at a glance:
- Where is the report coming from?
- Which type of Mac?
- What hardware is onboard?
- Which OS version?
- Which test ran; the Mac Driver or the X11 driver?
My boxes are reporting from CodeWeavers, so "cw-" is the first piece of the new name. That will be consist from each box that comes from systems I setup. Then, the type of Mac. We have MacBook Airs, MacBooks, MacBook Pros, Minis, iMacs; we have a decent collection to choose from and setup. We decided we could summarize them fairly easily with "mba-, mb-, mbp-, mini-, imac-".
Then, identifying the hardware. We needed to know something about the hardware. For us, the easiest and most informative way to describe the Mac seems to be the graphics card vendor and number of the most advanced card in the system: "a###-, i###-, or n###-"; as in AMD, Intel, Nvidia.
We needed to know what version of the Mac operating system was in use. This is simply expressed as the .## version. Mavericks is 10.9 so it would be 9, Yosemite is 10.10 so it would be 10, Snow Leopard is 10.6 so it would be 6.
Finally, we have the option to run with the Mac Driver or the X11 driver in these tests. We left this simple, either "m" or "x".
Now, the same system above is expressed in the test results pages as:
- cw-mbp-A370M-11-m
- cw-mbp-A370M-11-x
This suggests I got X11 to run with WineTest in addition to the Mac Driver. This is true. Getting those results to run is as easy as adding some lines to the configuration file:
email="caron@codeweavers.com" tag="cw-mbp-A370M-11-x" tagmac="cw-mbp-A370M-11-m" desc="OS X 10.11 dual GPU AMD Radeon R9 M370X and Intel Iris Pro with the X11 driver" descmac="OS X 10.11 dual GPU AMD Radeon R9 M370X and Intel Iris Pro with the Mac driver" excludemac="windowscodecs:tiffformat windowscodecs:converter user32:msg gdi32:dc dwrite:font"
There is a set of lines for running the Mac Driver and for running the X11 driver. It means that wt-daily will run two sets of tests, back to back. Sadly, this will not be enough to run WineTest on the X11 driver because X11 crashes during a run on several of the ddraw tests and in user32. This is consistent on all of the Macs we've setup to date and with the crashes, WineTest cannot continue, so we skip those tests with an exclude line:
email="caron@codeweavers.com" tag="cw-mbp-A370M-11-x" tagmac="cw-mbp-A370M-11-m" desc="OS X 10.11 dual GPU AMD Radeon R9 M370X and Intel Iris Pro with the X11 driver" descmac="OS X 10.11 dual GPU AMD Radeon R9 M370X and Intel Iris Pro with the Mac driver" exlude="ddraw:ddraw1 ddraw:ddraw2 ddraw:ddraw4 ddraw:ddraw7 user32:win" excludemac="windowscodecs:tiffformat windowscodecs:converter user32:msg gdi32:dc dwrite:font"
With those tests skipped, WineTest will successfully run both the Mac Driver and X11 driver.
This still isn't enough for a beer... because I'm human and I'll forget to run the tests. It is enough to get results on the test page.
About Caron Wills
Caron has been working in the computer software industry for over 10 years. She joined CodeWeavers in 2008 and became the Quality Assurance Manager for CodeWeavers in 2009. Contact Caron at caron@codeweavers.com and learn more about her professional accomplishments on LinkedIn.
About Caron Wills
Caron has been working in the computer software industry for over 10 years. She joined
CodeWeavers in 2008 and became the Quality Assurance Manager for CodeWeavers in 2009. Contact Caron at caron@codeweavers.com and learn more about her professional accomplishments on LinkedIn.
