Halloween!
Had some great fun giving candy out to the kiddies in the neighborhood. Also put up my usual window display – which got rave reviews!
Had some great fun giving candy out to the kiddies in the neighborhood. Also put up my usual window display – which got rave reviews!
I just spent an hour or two tracking down the last 16-byte memory leak in a project that I was working on. What it turned out to be was actually something EXPLICITLY cited in the Microsoft reference manual as having been fixed. Yet it turned out not to be.
I had finished instrumenting my program for CRT memory reporting. CRT memory leak reporting is normally very handy as it tells you exactly what lines of code are the source of the leak, how many bytes, and dumps the data that’s leaking. All it requires is that you instrument your headers as described here <http://msdn.microsoft.com/en-us/library/x98tx3cf.aspx>
// this goes before ANY other includes in ALL your headers #define _CRTDBG_MAP_ALLOC #include <blah> #include <blah> ...main() { _CrtSetDbgFlag ( _CRTDBG_ALLOC_MEM_DF | _CRTDBG_LEAK_CHECK_DF ); // tells leak detector to dump report at any program exit _CrtSetBreakAlloc(18); // set breakpoint on a specific allocation number }
With all this done you can run the program and then see the results of the report in the Visual studio output window when the program exits:
Detected memory leaks!
Dumping objects ->
C:PROGRAM FILESVISUAL STUDIOMyProjectsleaktestleaktest.cpp(20) : {18}
normal block at 0x00780E80, 64 bytes long.
Data: < > CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD
Object dump complete.
If you use the _CrtSetBreakAlloc() with the allocation number ({18}in this case), when the program starts up in debug mode the debugger will automatically stop on that allocation. You can then look at the call stack and get a great idea of whats leaking and why. Overall, it’s very handy and helped me track down all my leaks.
Well, the problem was that I fixed all the leaks but 2 in my particular program. I reduced the code down until I was down to just main() that dumped the memory report and exited. The problem was that it STILL reported a leak! I added a fake leak, and the very first allocation (that was leaking) in the program had a higher allocation number than the allocations that were supposedly leaking. This implied quite strongly that the leak is happening BEFORE I actually even get into the program. How is that possible?
Well, there are a few things that get allocated for you before your program starts. Fixed-sized arrays, (int array[10];) string literals, and static variables are all objects are allocated on the stack (as opposed to the heap) are usually allocated at load time. When I looked at all the static allocations in my program, I saw this:
class Bar
{
...
private:
static std::string m_name;
static std::string m_value;
}
Do you see any problem? I didn’t – at first. The problem is that std::string is a STL object that has a constructor. Statics are usually allocated at load time. Which means their constructors are likely being called during load time BEFORE the main() is hit. We assume that the corresponding destructors are called after main exits. Apparently, the CRT memory tracking starts right as the program starts but is supposed to ignore any CRT allocations (ones made by the standard C libraries). Apparently these static allocations by STL objects are captured. Since we report leaks BEFORE we hit the program exit, we haven’t actually hit those ‘hidden’ destructors for those static strings yet. Those allocations are still reported as active – and hence ‘leaked’. I only caught this because I’d initialized those strings with string literals. The leak dumped the memory and I recognized the strings. So a good way to test if this is happening to you is set your object data to something unique and make sure those unique values appear in the leak report memory dump.
Yet another clue is that the value of an uninitialized string consists of a number that appears to be a memory address (while the addresses change each run, other allocations in that run have roughly the same addresses), and a bunch of zeros. This likely corresponds to a pointer to the character buffer, and an int that indicates size. This is further confirmed by the fact the leaks are 8 bytes each in x86 build, and 16 bytes on a x64 build. This corresponds to the sizeof(int)+sizeof(void*) sizes on the respective builds.
Solution:
The solution is to ignore what is (likely) a false report, or change the static std::string objects to non-static or static std::string* pointers. Then make an appropriate initializer/destructor that creates/destroys those string objects. This means the memory is allocated/deallocated on the heap in the scope of the program run. The other thing you do is file a bug with Microsoft and put a few comments on the documentation website that indicates the statics are still leaking and they haven’t really fixed it yet. 🙂
Links:
MS article about CRT memory debug operations that says this bug had been fixed (but it has not as of VS2012)
Sometimes more really isn’t the answer.
Case and point – dishes at your house. I saw this picture and college days of yore flooded back:

At one bachelor flat I lived at – our sink often looked like this. We had 4 guys living in one house, and dishes weren’t a high-priority item. So, they tended to pile up and then nobody seemed to remember who dirtied what and an endless cycle of blame and avoidance resulted.
The solution: throw out all the extra dishes. Instead, there was *exactly 1* set of silverware, plate, and glass for each person. Further, each one was color-coded. Red plate/cup/silverware was Bob’s. Green was Fred’s, etc. That meant if you wanted to eat something, then you had to have washed your stuff to do so. It also meant if you left your dirty dishes around – the culprit was immediately identifiable. The first benefit was that with only 4 plates in the house, a ‘pile’ of them never got very big. The second was that very quickly the innocents were vindicated, and slobs were identified. (We still had a full set of ‘nice’ dishes for when people came over for a dinner – but they were in a separate cupboard. Anyone seen using those individually would be berated furiously). The system worked like a charm and there was nary an argument about dishes for the rest of the time I lived there.
I think this is a corollary to many a home-owner’s observation that your possessions naturally expand to fit the space you have. Or maybe Apple’s design philosophy that sometimes its the limitations you put on the usage of the device that actually allow easier and higher functionality.
I hadn’t taken much vacation this year (3 days to be exact), and had been grinding on a lot of code the last few months. So, I dug through our list of corporate discounts and found a great place on the Oregon coast I could stay at for cheap. So, I took 2 days off and went to Pacific City, OR and got this great view from my room every night.
I feel I got quite lucky because the weather at the coast isn’t usually very clear in the fall – but I got two near-perfect days. Guess someone was looking out for me. 🙂
Summary:
Very good trip, but not a huge ‘groundbreaking’ year of new technologies.
Sunday:
Course – Destruction and Dynamics for Film and Game Production
This all afternoon session started with a great overview of current destruction techniques. There was a great and thorough overview of the current ways to break up a mesh (Voronoi, Boolean, FEM convex decomposition, and tetrahedral creation), the methods and problems of collision detection after the breakup, and constraint solving. The focus now is not so much on the math of the constraint solvers, but on how to use them in intuitive and fast ways for the artists.
Takeaways:
Monday:
Course – Advances in Real-Time Rendering – Parts I & II
http://advances.realtimerendering.com/
–
Tuesday
Course- Beyond Programmable Shading I + II
All day course that talked about the direction and emerging graphics trends. I alternated between this and looking at posters during the day.
Poster sessions was pretty good with lots of entrants. Nothing ground breaking, but there were several topics that I now want to look up. (don’t have notes with me today).
Wednesday:
Thursday:
A grab bag day of talks as the all-day courses were mostly over.
Hiding Complexity (occlusion culling):
Smokin’ Fluids:
Discrete differential geometry
Filtering Techniques for AA
Battlefield: Bad Company 2 on my PC was causing me MAJOR grief. I’d get into a game, and about 5 minutes in, I’d get dropped and ‘connection to EA server lost’ message. UTTERLY annoying. Appears tons of other people have the same problems. Various solutions from turning firewalls off/etc – but that didn’t work for me.
Changing your router’s UDP and TCP endpoint filtering to “Endpoint Independent” certainly helped – but it would still drop after about 10 minutes.
What seems to have worked is this. Go to your steam client and:
Steam Library -> rightclick BC2 -> properties -> Set launch options… -> “-pingInterval 50000”
Paste the “-pingInterval 50000” phrase without the quotation marks and that’s it.
That seems to have solved the problem for me; but this was pretty much utter fail on EA/Dice’s part. The amount of hate-forum mail that’s gone on for over 2 years relating to this problem is amazing.
So, when playing the AIX mod for Battlefield 2, I kept getting kicked after about 2-3 minutes of play when punkbuster(!) complained that:
PnkBsrtrB.exe handshaking failed ; PnkBsrtrB.exe heartbeats stopped ; PB init failure ; Packet loss
The basic problem boiled down to the fact that when Punkbuster installed, it created the outgoing rules in windows firewall, but it did not create the INCOMING rules. I simply added incoming rules for:
c:windowssysWOW32PnkBsrtrA.exe
c:windowssysWOW32PnkBsrtrB.exe
And set them to ‘allow’. That turns out to fix the problem.
I still can’t believe how horrible the solutions to prevent cheating were back in the day.
Woot! I just set up my system the last few nights with *three* 2TB drives all connected in RAID 5 setup into a single, 4.0 TB partition – AND I am using that huge bit bucket as my primary boot drive. Now I have speed from the data striping and redundancy from the RAID. It’s quite nice so far, but I’m still getting all my old software installed. However, up until about a month ago – this wasn’t possible with the standard Intel storage controller that comes with almost all the new 2nd generation core systems (or with most RAID controllers). But getting this magical land to work involves a lot of new information and a good bit of bleeding-edge fighting.
What you need to know:
First off, getting a RAID-ed boot partition larger than 2.2TB involves several pieces that need to work together. If you only have the first 2, then you can still boot from a single-drive larger than 2.2TB, but you won’t be able to use it in a RAID setup with large partitions.
First off, you need a motherboard with an EFI (Extensible Firmware Interface) bios. EFI bios is the successor to the 20+ year old CMOS bios. You might remember or have seen CMOS when that awful text screen came up when you hit the power and it would count memory or say ‘Press [del] to enter setup’. Well, Intel with a lot of other companies finally, happily, heavenly, triumphantly are putting the nails in what had become a painful bit of computing legacy by throwing out CMOS and inventing EFI. Apple has had EFI bios in their machines for a few years now, and some of the first-generation Core i3/i5/i7 motherboards had EFI support too – but it was spotty and very few things seemed to take advantage of it. Which brings us to #2 – needing an OS that supports EFI booting and GPT.
If you have a motherboard that supports booting in EFI mode, you still need an OS that supports it and knows how to boot from partitions created with GPT (GUID Partitioning Table). I had an EFI board on my old system, but you couldn’t actually boot in EFI mode unless you used an early Linux distribution with EFI support or until about Windows Vista SP1/Win7 range. Not only that, but EFI booting only worked on 64-bit versions of the Vista/Win7. If you’d bought the 32-bit version – then you were out of luck. The Vista route was also painful because unless you had an install disk that had the EFI boot files on it, you couldn’t actually install Vista in EFI mode. Early versions of the Vista install CD’s (at least the one I had) didn’t have the EFI boot files. I never learned if someone who had one of those early disks could return it and get an install disk with EFI boot support. The reason you needed this is because if you couldn’t boot in EFI mode, you were stuck with the 2.2TB CMOS MBR boot limit. If you weren’t in EFI mode during the install process, Vista/Win7 will refuse to create a boot partition larger than 2.2TB because it knew a MBR system couldn’t boot from it. Unfortunately, you couldn’t override this in the install. GPT booting was usually only supported when the BIOS booted in EFI mode. So if your OS install disk couldn’t boot to EFI, or wasn’t able to make GPT boot partitions in CMOS mode, you were still out of luck. As you’ll see, booting in EFI mode is actually more than just having a disk with the EFI files on it. It’s a two-step approach, but those details will come later.
Finally, Windows 7 64-bit comes along and seems to solve our problems. The OS install disk has EFI boot support on it. The install disk is able to make bootable GPT partitions larger than 2.2tb. Awesome. But until this last month, you weren’t able to get your Intel RAID controller to make a bootable RAID set larger than 2.2tb because Intel hadn’t finished writing the EFI bios support for it. You have to check the motherboard manufacturer’s website for a BIOS update – but you should now see new BIOS’s with the new large-partition RAID booting support. That was the final key to the puzzle.
So, without further adieu, here is how one does this with the following equipment:
I know many Linux afficianatos cringe at big bit buckets like this. Put your swap on another partition! Make a partition for your programs files! Make a tiny one for your boot partition! This is all well and good if you like micro-managing. I hate micro-managing. I hate extra drive letters. We’re in the f-ing 2000’s now people! 3TB drives are $150, and 2TB drives are $65. RAID comes built into boards for free. Storage is a commodity.
I use Windows for most of my daily game playing/tv watching/etc and don’t want to futz with the headaches of multiple partitions. I want to play games, watch tv, and surf the web. I don’t want to have to worry about how big to make a partition for my programs files or data files or worry about how big my boot partition is on windows. What if I over or under-guess? I get to either re-install or test my luck with a partition resizer or have some programs one place, and others somewhere else. Or the fun of every. single. time. I install a program, I have to select a custom install and pick a different drive letter. Windows patches often automatically install in the system32 directory – what if a new service pack is too big to fit on my tiny boot partition? It makes no difference for security if my boot is on a different partition than my data or programs. Creating a separate partition on the same drive just for the swap file is no gain. Unless it’s on another physical drive, you’re not gaining any speed in Windows. And even then, I have 16GB of memory in my machine (and you can too – it’s ridiculously cheap at $60/8GB of high-quality DDR3 ram), I never swap anymore.
In short, I never have to worry about my space or what I’m putting where. We’re not in the 70’s anymore where drives were real investments. They’re cheap, replaceable commodities now. I have a RAID 5 which means I’m getting the best of both speed and redundancy all in one place, and it’s details are taken care of by the hardware. What more could one ask for.
Did an ‘easy’ hike (compared to Hood anyway) up Dog Mountain this weekend, and man was it windy. Flowers on the upper peak were in bloom and beautiful, but man was it windy! 20-30mph sustained.
After make a successful climb of Hood, here are some of the helpful websites I used in planning:
Cascade Climbers – message board for folks that just did climbs/etc
http://cascadeclimbers.com/forum/ubbthreads.php/forums/7/1/Oregon_Cascades
Mountain Forcasts.com – Best source of wind/freezing levels/temperatures by elevation for many summits:
http://www.mountain-forecast.com/peaks/Mount-Hood/forecasts/2500