OpenAI announced on Tuesday the release of its newest picture-making machine POINT-E, which can produce 3D point clouds directly from text prompts. Existing systems like DreamFusion typically require multiple hours — and GPUs — to generate their images, Point-E only needs one GPU and a minute or two. Paper here, and code here.
Links:
You can use noise dampening curtains, or maybe even try this handy window inserts that use rubber edges to pop in and out of your window frames easily.
Epic games worked with Quixel and just release the photogrammetry app RealityScan which turns smartphone photos into high-fidelity 3D models. The app is called RealityScan and is a paired down version of the desktop version of RealityCapture. Both combine a set of 2D images to make 3D assets. The idea is to enable game developers and other creatives to scan real-world objects at any time and any place for their projects (or the MetaVerse if that becomes a thing).
Engadget tried the iPhone app out and shows us how it works.
The scanning process begins with signing into your Epic Games account and taking at least 20 photos of an object from all sides. As you move your phone around, a real-time quality map shows how well you’ve covered it: green denotes well-covered areas, yellow could use more attention and red needs the most additional photos.
The app uploads and automatically aligns the images in the cloud as you take the photos. You can preview the model through the camera view and switch between the quality map and an in-progress, full-color render. When you want to crop it, it pops up 3D handles for you to drag around, ensuring it captures only the item, not the floor beneath it or background objects.
The process works best with simple items captured in even, indirect lighting (reflective or wet surfaces won’t capture well). It also appears to work best with larger objects, as my attempt to capture a small Mr. T action figure resulted in something that looks more like a pointillistic painting than a usable model.
The iPhone App version hasn’t got very good reviews (2.2 stars) – but it’s a start.
This idea isn’t new. There has been research and experiments in this space since the early 2000’s; but it’s an interesting attempt even if it seems to have a lot of growing pains to work out.
My very first computer was a TSR-80. I learned to first code out of necessity – you had to type in all the programs you wanted to play. My dad then purchased one of these wonderful beauties: The IBM PC 5140 Convertible.
The computer was a miniaturized IBM XT 8088 system with only 2 floppy drives and no hard drive. Still, it was a huge step up with the ability to save and load programs from floppy disks.
I love that people try to preserve the history of these wonderful old machines – and you can find everything about the system including original boot disks, specs, and configurations. There also appear to be copies of the original boot disk and ROM set that work on MAME.
DALL-E 2 beta now provides an open API which allows users to embed the ability to generate new images from text prompts or edit existing images in their own apps.
Microsoft is already leveraging it in Bing and Microsoft Edge with its Image Creator tool, which lets users create images if web results don’t return what they’re looking for. Fashion design app CALA is using the DALL-E 2 API for a tool that allows customers to refine design ideas from text descriptions or images, while photo startup Mixtiles is bringing it to an artwork-creating workflow.
Pricing for the DALL-E 2 API varies by resolution. For 1024×1024 images, the cost is $0.02 per image; 512×512 images are $0.018 per image; and 256×256 images are $0.016 per image. Volume discounts are available to companies working with OpenAI’s enterprise team.
Link:
There have been numerous attempts to make Stable Diffusion plugins for Adobe Photoshop. Many are not well baked nor free at this point but some are worth taking a look at.
KerasCV is another implementation of stable diffusion. It offers some interesting features that makes it one of the fastest implementations of stable diffusion:
jit_compile=TrueCheck out this setup, feature walkthrough, and comparison guide.
You can get stable diffusion to work on the new Mac M1 (and M2’s). Above is me getting stable diffusion running on my Mac Mini M1 with 16gb of ram (using these instructions).
It’s definitely not as fast as dedicated GPU’s, but it does have the advantage of not running out of memory like you will with graphics cards that only have 8gb or 16gb of memory. Most GPUs require at least 24gb of ram to run 512×512 stable diffusion images (without using lower resolution models). I was able to generate 512×512 and 768×768 images with my 16gb mac mini m1
This is one of the really big advantages of a unified memory architecture in which the CPU and GPU can access the same memory without needing to be transported across a PCI bus first.
Links
What are the fastest/best growing jobs for entry-level jobs?
This study by Indeed looked at entry-level roles (with an average salary of more than $40,000) to find those with the strongest demand across the US. Entry-level jobs were defined as those require no more than three years of experience, and demand was measured by assessing the growth in the share of postings for each role from September to February compared with the same period the year prior.
The top 3 college jobs are not nearly as sexy as years past (sales representative, transportation coordinator and quality auditor) and the average salary for the college grad was $69,000. For non-grads the average salary was $54,200.
As has been noted by many, trades make some great career paths now with faster-than-average rising salaries and high demand, while college degrees might have reached oversaturation in some fields. A problem that some see as one of the causes of recent sociopolitical stress as many educated people cannot find work in their field and turn to protests/political disenfranchisement.
Here’s the breakdown summarized on Bloomberg:
First we had simple cut-and-paste. Then intelligent selection cut-and-paste. Then content aware cut-and-paste. Now we have select-and-replace with completely auto-generated art via Pair-Diffusion. Amazing times we live in.