Browsed by
Category: Technical

Multi-core compiling in Visual Studio

Multi-core compiling in Visual Studio

You might already know this, but this is for those of you that want to compile extra-fast on your multi-core beast.  Bet you didn’t know that by default, most versions of Visual Studio do NOT use multi-core compiling.  So, to turn it on, do this in visual studio:

Tools > Options > Projects and Solutions > Build and Run > maximum number of parallel project builds

Set this to the number of cores you have (or the number of cores you have -1 if you want to do things on your desktop while compiling extra-big things).

To see if it’s working, when you compile, in the compiler output window at the bottom you should see each line prefixed by a number like this:
1>blahblah
4>blahblah
3>blahblah

Those prefix numbers tell you which ‘core’ the message is coming from.  I find this speeds up your compile times dramatically – especially on large projects.  Give it a try!

Why the volatile keyword probably isn’t necessary in multi-threaded programming

Why the volatile keyword probably isn’t necessary in multi-threaded programming

Interesting article from the Intel guys doing TBB.

Arch Robinson just removed almost ALL the volatile keywords from Intel Thread Building Blocks.  Why?  For several reasons, but mostly because he claims that overall it slows your code, probably does not actually solve the underlying ordering problems if your code needs to be portable (a REAL concern on today’s writing of games/apps for x86, Xbox, PS3, and iPhone devices!),  and likely isn’t doing what you think it’s doing anyway.  Here’s a pertinent example:

Sometimes programmers think of volatile as turning off optimization of volatile accesses. That’s largely true in practice. But that’s only the volatile accesses, not the non-volatile ones. Consider this fragment:

    volatile int Ready; 

    int Message[100];

    void foo( int i ) {

        Message[i/10] = 42;

        Ready = 1;

    }

It’s trying to do something very reasonable in multi-threaded programming: write a message and then send it to another thread. The other thread will wait until Ready becomes non-zero and then read Message. Try compiling this with “gcc -O2 -S” using gcc 4.0, or icc. Both will do the store to Ready first, so it can be overlapped with the computation of i/10. The reordering is not a compiler bug. It’s an aggressive optimizer doing its job.

You might think the solution is to mark all your memory references volatile. That’s just plain silly. As the earlier quotes say, it will just slow down your code. Worst yet, it might not fix the problem. Even if the compiler does not reorder the references, the hardware might. x86 hardware will not reorder it. Neither will an Itanium(TM) processor, because Itanium compilers insert memory fences for volatile stores. That’s a clever Itanium extension. But chips like Power(TM) will reorder. What you really need for ordering are memory fences, also called memory barriers.

So what’s the solution for multi-threaded programming? Use a library or language extension hat implements the atomic and fence semantics. When used as intended, the operations in the library will insert the right fences. Some examples:

* POSIX threads
* Windows(TM) threads
* OpenMP
* TBB

So, when is volatile actually necessary?  It turns out there are only 3 portable cases volatile is actually needed:

  • marking a local variable in the scope of a setjmp so that the variable does not rollback after a longjmp.
  • memory that is modified by an external agent or appears to be because of a screwy memory mapping
  • signal handler mischief

And now you know, and knowing is half the battle.

Good presentation on PS3 hack

Good presentation on PS3 hack

An international group recently hacked the PS3 so that Linux could be run on the platform and recently presented how it was achieved at a security conference.  Some may recall that the PS3 originally had this option (called “OtherOS” boot), but then the feature was removed after Sony claimed it allowed cracking of their games/piracy concerns.  A bios patch was forced from Sony and no machine without the updated BIOS is allowed to use their services.

At this point, the group ‘failOverflow’ then picked up the mantel for angry PS3 users that had bought the console with the goal of running Linux on it and then felt they’d gotten cheated by Sony who was viewed as having reneged on their promise.  After about 12 months of work, a hack was achieved.

I gleaned two interesting things from the presentation.  Firstly, successful hacks of such modern devices usually comes from teams, not individuals.  While working together is obviously a logical progression if several people are trying to hack the same, much more complex device; it does seem to be a big change from the days when a single guy in his garage would ‘prove’ himself by hacking something by themselves.  Secondly, these guys are very smart.  They clearly have very high levels of understanding of hardware, memory architectures, operating system concepts (loaders, ring levels, decryption, trust-chains, etc), and software stacks.  I’m almost certain they all have a Computer Science or similar background.  The days of a single guy picking up a book, a debugger, and hacking the security in these consoles in their spare time seems to have come quickly to a close.  I think this trend started with the hacking of the original Xbox by a team of Computer Science grad students (which took advantage of an awesomely obscure memory wrap-around bug introduced when they switched from AMD to Intel at the last minute), and this trend doesn’t appear to be going back.  It appears that if you want to contribute to hacking a platform; you best get your BS/MS in CompSci or CompEng.

So, without further ado, here’s the video clip of failOverflow talking about their efforts (along with an interesting bit at the beginning on how long it took to hack other platforms)

Battlefield 2 on Steam kicking you off because of PunkBuster?

Battlefield 2 on Steam kicking you off because of PunkBuster?

Did you take advantage of the super Steam holiday sales?  I did, and got the complete Battlefield 2 pack for $4.99.  However, if you installed it and got it all working – you might keep getting kicked out of network games because PunkBuster reports that you have an ‘Invalid driver version/game’.  This is because of the Steam Community overlays.

  1. At your game library list in Steam, just right-click the Battlefield 2 icon
  2. Select Properties
  3. Go to the General tab
  4. UN-select the ‘Enable Steam Community In-Game’ option.
  5. Voila

Yes, annoying.  Can’t believe old games like this haven’t been re-modified by steam to not need the now-ancient PunkBuster system.  The fact they’ve had to update PunkBuster to be separate Windows Services now in order to work shows you it’s time to see that stuff go.

Adobe registration box always keeps popping up

Adobe registration box always keeps popping up

If you have any of the Adobe suite and continually get the registration box when starting the app (even if you’ve filled it out 10 times before or told it not to register) you’re in good company.  You just need to start the app in Administrator mode, make you registration/hit the don’t register button, and THEN the registry can be properly updated.

Thanks again UAC for messing up my life in weird, cryptic ways that not one grandmother would ever be able to figure out.  But I’ve ranted about this before…

The key is MOV EDI, 0×9C5A203A

The key is MOV EDI, 0×9C5A203A

That’s the assembly instruction you need to unlock a secret ‘debug mode’ on AMD processors since the Athelon.  While you need to be in ring 0 to execute it; it did bring up some interesting possibilities of using the special debugging mode for reverse-engineering operation of the chip, accessing possible new features, or presenting a chink in the security armor.  So far, the security problems don’t seem to be probable, but if they cause undocumented resets/etc – they might be.

Anyway – interesting article.  Original posting here.

Cartalk conundrum

Cartalk conundrum

The guys from Car Talk have weekly puzzlers, but this question wasn’t a puzzler, but this came from a truck driver who called in.  He said (basically):

“I have big cylindrical tanks on my truck that lays sideways under the foot step.  Problem is that my gas gauge is broken.  I have a stick that I can put in vertically, so if the gas is at the 20″ mark on the stick, it’s full.  If the gas reaches the 10″ mark on the stick, the tank is clearly half full.  Where should I put the 1/4 and 3/4 marks on the stick?”

First you’d think they should be at 5″ and 15″, but that’s not right because the tank is round, which means the bottom and tops have less volume per inch of height.   Then you think this is a problem is an integration problem – which it can be – but the integration becomes extremely hairy.  Then, you find you can back up and use a geometric method (and when you can’t reduce anymore) use a numerical method to solve it.  So let’s get started!

We see that needing the actual volume of the cylinder is unimportant since you can solve this problem with just the cross-section – which is a circle.  What you want is a circle with a chord across it in which the volume between the chord and the outer wall is 1/4 the capacity of the circle.  So, you draw a diagram, and get started!

Unfortunately, you see that the equation becomes very difficult to solve analytically – and one must resort to numerical methods to get an actual solution.  I used the Mathematica online site, but you could easily use the Newton-Raphson method as well.  Whatever way you use, you find that he should mark the 1/4 tank line 5.96027 inches from the bottom of the stick.  3/4 and 1/8th values are also shown.

The value of this equation can quickly be used to calculate 1/8, 1/16, and all other desired fill marks by simply changing the 1/4 * pi * r^2 line whichever fraction you’d like. In fact, you can graph it to get any value:

Ignoring negative volumes, you see that the tank’s volume compared to it’s theta (roughly equivalent to height) forms a S curve, so that you can see that the height changes more rapidly w.r.t. volume when close to full/empty than in the middle – just like we’d expect.

So, that’s your answer.  Turns out, others have solved this since it’s a common problem with all kinds of other tanks (fuel oil, gas stations, etc).  Here and here are other solutions that verify the same process and confirm that the final equation is unsolvable analytically.

Another person pointed out that most semi’s have TWO tanks – one on each side – which are connected by a balancing flow connector.  So both tanks fill and empty evenly.  Even though this seems to mess up the problem, it actually does not.  In order to represent that situation, you simply multiply both sides by two (two tanks, two times the target volume) – which cancel each other out.  You could have ANY number of tanks connected like this and the answer is the same.

It also doesn’t matter how long the tank is either (so long as the tanks are the same size if you have more than one).  Finally, the theta angle you calculate doesn’t even depend on what radius of the tank!  So if you calculate the thetas for all the fill points, then you can calculate the 1/4 mark on ANY size tank.  Pretty nifty huh.

error LNK2019: unresolved external symbol _vmlsLn4

error LNK2019: unresolved external symbol _vmlsLn4

If you get this message while converting some code from compiling with the Intel Compiler over to using Visual Studio, then I have a solution for you:

error LNK2019: unresolved external symbol _vmlsLn4 referenced in function “BLAHBLAH”
fatal error LNK1120: 1 unresolved externals

Problem:
The Short Vector Math Library (SVML) which has the vmlsLn4() function is implemented in  svml_disp.lib.  Now this lib is usually only for the Intel Compiler’s use in vectorizing code but if someone used any of the SVML instructions, you can do the below trick to fix it.  See this article for more details about using SVML.

Solution:
1. Install the latest Intel Compiler (with Visual Studio links if it asks).  If that doesn’t come with the library you need, install Install Intel Parallel Studio as well. Between the two, you’ll get the svml_disp.lib you need.

2.  At the top of the .h/.cpp file that uses , declare this external:
extern
 “C” { __m128d vmldLn2(__m128d a); }

3. Go to your project settings, and for the library includes, add:
svml_disp.lib

4. In the ‘Additional Library Directories” box, add a path to wherever you installed the Intel compiler’s lib directory.  The default location is:
C:Program Files (x86)IntelCompiler11.167libia32

Hit F7 in Visual Studio and you should build like a champ and run just fine.