Using a new language model that is trained from scratch
The training dataset was heavily censored with a NSFW filter
The second part would have been fine, but the filter was quite inclusive and has removed substantial amount of good-quality data. 2.1 promised to bring the good data back.
Stable Diffusion was developed by CompVis, Stability AI, and LAION. It mainly uses the English subset LAION2B-en of the LAION-5B dataset for its training data and, as a result, requires English text prompts to producing images.
This means that the tagging and correlating of images and text are based on English tagged data sets – which naturally tend to come from English-speaking sources and regions. Users that use other languages must first use a translator from their native language to English – which often loses the nuances or even core meaning. On top of that, it also means the latent model images Stable Diffusion can use are usually limited to English-speaking region sources.
For example, one of the more common Japanese terms re-interpreted from the English word businessman is “salary man” which we most often imagine as a man wearing a suit. You would get results that look like this, which might not be very useful if you’re trying to generate images for a Japanese audience.
rinna Co., Ltd. has developed a Japanese-specific text-to-image model named “Japanese Stable Diffusion”. Japanese Stable Diffusion accepts native Japanese text prompts and generates images that reflect the naming and tagged pictures of the Japanese-speaking world which may be difficult to express through translation and whose images may simply not present in the western world. Their new text-to-image model was trained on source material that comes directly from Japanese culture, identity, and unique expressions – including slang.
They did this by using a two step approach that is instructive on how stable diffusion works.
First, the latent diffusion model is left alone and they replaced the English text encoder with a Japanese-specific text encoder. This allowed the text encoder to understand Japanese natively, but would still generate western style tagged images because the latent model remained intact. This was still better than just translating the stable diffusion prompt.
Now Stable Diffusion could understand what the concept of a ‘businessman’ was but it still generated images of decidedly western looking businessmen because the underlying latent diffusion model had not been changed:
The second step was to retrain the the latent diffusion model from more Japanese tagged data sources with the new text encoder. This stage was essential to make the model become more language-specific. After this, the model could finally generate businessmen with the Japanese faces they would have expected:
CppCon has a bunch of other great ‘Back to Basics’ videos that cover a whole host of great topics: safe exception handling, move semantics, type erasure, lambdas, and a bunch of other critical but oft misunderstood elements of C++
In this video, you get a refresher on RAII.
“Resource Allocation is Initialization is one of the cornerstones of C++. What is it, why is it important, and how do we use it in our own code?”
Interviewing seems to be changing. Besides the usual technical and behavioral interviews, I’m hearing of more interesting two-hand kind of tests in which recruiters ask you to basically perform some tasks as if you were going to be doing the job.
One of the critical arguments during the purchase of Twitter was how many accounts were bots. Bots are probably much more rampant in social media and games than people think – or would like to think. With things like ChatGPT, people are quickly realizing how easy it has been for bad actors to flood social media (from reddit to Facebook) to disrupt our elections and foment rebellions (and protests) in countries to destabilize governments (include in the US).
Smarter Every Day did an entire series on how easy it is to manipulate everything from Facebook to Youtube to Reddit. Reddit even has whole forums that are just bots talking to each other. Go read through it – it’ll shock you how easily bots create and control conversations on these platforms.
How easy is it to get followers and spread misinformation? What if I told you that you could get 1000 followers for $20? If you have a modest sized advertising budget (or part of a country’s defense/espionage budget), you could spend $5,000 and get 250,000 followers or only $20,000 and get a MILLION followers. This is true for Instagram, Facebook, Twitter, Youtube, and just about any other platform. Don’t believe me? Here’s just one example from BoosterGod:
So, maybe being an ‘influencer’ with a million subscribers isn’t quite as impressive as you’d think. That isn’t even a blip on most corporate advertising budgets – and nothing to a foreign power.
It might be worth an experiment if you have the cash. An influencer with 1 million subscribers can usually get advertising sponsorships and product advertising deals that far exceed $20,000 – so it might even pay for itself…
Game development is now as much art as science, or rather the art of science. Even something as simple as how and when to use randomness can profoundly impact the fun of a game. Enter the observation of two different kinds of randomness: input and output randomness.
Input randomness is randomness that is decided BEFORE a player makes their strategy and decisions. Examples would include having a random number of enemies generated before the fight starts. While the number is random, knowing how many will show up actually lets the user decide to use different strategies and feel more in control.
Output randomness is often a big contributing factor to frustrating parts of gameplay. Examples here would consist of attacking an enemy, only to find out your attack completely missed out of sheer bad luck or an usually bad hit roll. This kind of behavior, while mathematically correct, often leaves users feeling like they were ‘robbed’ and that the game is cheating.
Games are increasingly using input randomness as a way to give users control. Even games that rely on output randomness often put their thumbs on the scales so that you do not lose as often as you’d like. In Civilization, if your unit with a 33% chance of hitting misses twice in a row, it’s guaranteed to hit on the 3rd try – even though real randomness wouldn’t behave like that.
Anyway, this is a great video about the different kinds of randomness.
Using a Neural Net as compression for character animation
This was published in 2018, but it’s a fascinating dual purpose use of neural nets. Firstly, there was a massively increasing issue with character animation. Character animation is quickly becoming highly complex as it has becoming more realistic. The problem compounds when you want to make sure you can do things like crouch and aim at the same time. Or crouch and walk across uneven terrain while looking left or right. You can imagine all the different kinds of combinations of motion that must be described and handled. This all started taking massively more time to develop by artists; but even worse it was taking up more and more storage space on disk and especially in memory space.
Daniel Holden of Ubisoft wondered if he could use a neural net to not only reduce the combinations they had to handle into a net but also utilize the inherent nature of neural nets to compress data. It turns out he could – and he presents what he found in this excellent presentation.
8″ floppy drives are the earliest form of floppy drives connected to early minicomputers. By the time of personal computers, 8″ floppy drives had been replaced with 5.25″ floppy drives. But those 5.25″ (and later drives) were still often based on the Shugard interface. Adrian’s Digital Basement shows how he hooked up an 8″ floppy drive from a TSR-80 Model II to a 386SX computer – and gets it to boot! This is almost certainly something I want to try some day.
With the right set of curvatures, it’s possible to make a clear object project an image that’s not visible until light shines through it. Science educator Steve Mould explains the optical and mathematical properties of these uniquely engineered lenses. It turns out the problem has a lot to do with moving the minimal amount of dirt to build a structure and was studied extensively by mathematicians who called it optimal transport. These transport theory problems have a number of solutions and applications.
A similar effect can be created with mirrors and reflected light. Rayform specializes in the technique for a wide variety of luxury and architectural items.
