Neil Cassidy

Updated. A really bad memory leak has been fixed! See first post for download. I was preallocating a large arraylist to avoid dynamic expansion and associated heap fragmentation, but instead of doing this only once, I was doing it for every region of interest. So I was using the effect on some large non-contiguous selections this morning, obtained by using Segment Image and Magic Wand, and got some out-of-memory errors. I fixed it up immediately and did a little refactoring. It shouldn't run out of virtual memory anymore, and performance is greatly improved (although still not optimal). Other changes: - Effect has been moved to the "Color" sub-menu. - Analysis stage has been moved to OnSetRenderInfo method. The double-checked lock + memory barrier hack isn't necessary unless the effect is built in Code Lab. - Instead of scanning over the selection's bounding box and using the IsVisible test to determine whether a given pixel within the box is actually selected, it now scans over the regions of interest directly, which also improves performance.

I knew that anonymous functions existed, but could never really find a good excuse to use one in an object-oriented context. This is definitely a case where it makes a lot of sense! Thanks for the tip.

Sorry to anyone who's been waiting! At long last, here's a version that polls IsCancelRequested. I'll put the download in the first post in a second. Just a few lines of code added, and it should cancel much faster. I haven't tested it exhaustively, but I was able to cancel a 10-cluster run on a 6 megapixel image within about 7 seconds. The previous version takes far longer to cancel (at least 40 seconds) because it has to complete the entire clustering run before it gets a chance to "quit". This makes me very happy, and I hope it makes others happy too. Thanks very much for adding the property, Rick! I could probably improve the cancellation granularity even further by more careful placement of the polling code, but I'll save those improvements for the rewrite that I'm working on, which should have significantly better performance anyways... some tests that I ran a couple weeks ago were very encouraging. I'll try my best to get it done this weekend - most of the difficulties that I'm experiencing have to do with design stuff rather than the algorithm itself.

Thanks for this, Boltbait! It's useful in and of itself as a single consolidated way to make a number of adjustments, and the source code is a great example of how to compose effects (and by composition, I mean function composition).

Yeah, that's pretty much the idea, but (from memory) I think it attempts to ensure that the Laplacian matches up between the luma and chroma channels, rather than the gradient. And it uses YCbCr, which is a very popular linear color space - SDTV, HDTV, JPEG, and probably tons of other things use it, mostly because it facilitates downsampling of color information while leaving intensity information roughly unchanged.

These techniques are really interesting! I hadn't read the papers before. Sometimes I manually inpaint simple things by selecting the void, filling it with the average color of some small surrounding region, and then running a small-radius Gaussian blur over the void iteratively until it seems to have converged. This is roughly equivalent to solving the Laplace equation over that region with Dirichlet boundary conditions... it ensures that the levels are smooth over the inpainted region, but the solution doesn't really respect what's "going on" at the boundary. * That's about all I can remember about partial differential equations, so understanding the one described in the first paper is a little tough. The most I can figure out is that, since the image will be constant at the steady-state solution (∂I/∂t = 0), the gradient of the Laplacian of the solution will be orthogonal (dot product of the two terms = 0) to the isophotes (curves of equal luminance). So the Laplacian of the intensity surface will be constant along the isophotes... but I'm having a tough time determining what exactly this means in terms of how it determines the structure of the final image... best guess is that it enforces constant "sharpness" along the isophotes, so that a crisp edge doesn't fade into a soft one or something. * Incidentally, it also happens to be one of the obscure cases where differences between "true" and "triangular" Gaussian blurs become apparent!

This happened to me as well! I edited a photo in Paint.NET, sent a copy to my girlfriend (via Windows Live Messenger), and on her end the thumbnail still displayed the original image. Refreshing the view didn't change it. Forgot all about it until I read this. Couldn't fathom how it could have happened, I didn't even know that the thumbnail was attached to the image file itself. I can't remember exactly how I fixed it, but I know that I nuked the Windows thumbnail database at some point. You can do that by running "Disk Cleanup".

This sort of thing is impossible in general (lots of rants on the internet about the infamous CSI "Enhance Button"), but in this case, one can enlarge the image and then sharpen up the edges by using some sort of local or global contrast enhancement. Tanel made a "Local Contrast Enhancement" effect, global contrast enhancement can be achieved by playing around with the left-hand slider in the built-in "Levels" adjustment. There are probably other ways to do it, but I can't think of any off the top of my head.

I saw an FFT-based technique like that in an "image processing cookbook" that I found somewhere. It was intended for removing noise, by presenting the power spectrum as a canvas and allowing the user to zero out parts of it. Pretty cool. If you're interested in developing a simple inpainting plugin, you might start with a method due to Efros and Leung (c. 1999?) that resamples small texels. It's pretty slow, but it's simple, and yields excellent results for small gaps in my experience. I think one of the authors has a fairly comprehensive website about the method, including papers and links to related research. The only really heavy artillery you'd need would be a way to sample from a Gaussian distribution, but there are plenty of relatively easy ways to do that, particularly http://en.wikipedia.org/wiki/Box-Muller_transform. I was going to do something like this, but I had already worked with the Efros-Leung technique in the past, and wanted to try something different. I eventually gave up on the idea mostly because any conceivable interface would be tough to either develop or use.

OK, this didn't turn out as well as I had thought. I was thinking that template matching would be sufficient for finding the images to replace. Turns out that it has a pretty tough time resolving the precise location of the pattern, and it also runs very slowly. It might be possible to speed it up by working at multiple scales and pruning at the faster ones, but I'm not sure how accurate this would be. There are other methods for doing the recognition part, but some are patented (i.e. SIFT) and most are still quite complicated. I think I'll have to put this idea on the back burner for now, sorry about this! I have to figure some other things out before I can code up the whole thing. Time willing, I should be able to put something together eventually.

Perhaps you're looking for the "Transparency" plug-in that's part of BoltBait's pack? It'll allow you to change the opacity of the selected part of a layer. viewtopic.php?f=16&t=22819

BriDog, some page that I found (http://www.planetavp.com/modmaker/modmaker2.html) suggests that a tool called DEdit can import/export Lithtech textures or skins in TGA format, which Paint.NET is certainly capable of working with. I don't know what this tool is, seems to be a level editor. Perhaps you already have it. You might try starting there!

This is actually a pretty basic pattern recognition problem! Your idea is really interesting actually, I've never thought of running a "find and replace" operation on an image. I can certainly code up a plug-in that does this without much trouble. I have a couple of other things on my plate at the moment though, how soon do you need it?

Kris, I'm definitely not going to optimize this effect for the 800x600 images that I might be able to fit into a decent-sized L2 or L3 cache. I can't control what images users want to run it on, and so I should probably be making performance acceptable for the largest reasonable size, which is roughly on the order of the 40MB images that my camera shoots. Anything around that size can't possibly fit in L2 or L3, and it's pointless to try to make it happen. I'm quite interested in using the effect on images that large, actually. I'm never going to achieve that if I spend my time on tedious low-level optimizations. Thanks for getting me thinking about cache utilization though. I believe that I've come up with a suitable change to the underlying algorithm that will allow this effect to make full use of any cache, just have to implement it now. Major update coming soon!

It's a textbook actually! Imaging and computer vision stuff. I'd highly recommend borrowing it if you're interested in this sort of thing. This effect is not really intended to isolate objects... as I mentioned in the first post, it's supposed to cluster similar pixels together. The similarity measure is squared distance in a six-dimensional space that consists of color and location variables (and I might add texture descriptors too). There are much better ways to isolate objects, actually. Wikipedia has a pretty good description of the overall goal of image segmentation (and of many techniques) on this page. If I were to just use the color variables, this effect would perform color quantization, which is what you're thinking of. If I were to just use the location variables, the effect would return what's called a centroidal Voronoi tessellation (a tiling of the plane into cells based upon a set of points called generators, such that all points in any given cell are closer to their own cell's generator than to any other cell's generator, and each cell's generator is also its centroid). You can see something very close to either result by biasing the effect all the way towards color or location, respectively. Try biasing all the way towards location when operating on a manual selection - it will partition the shape into cells and fill the cell with the average color of its members. The color information still has a bit of influence at 0.99, so there will be a little "noise" in this version. Example: If you play around with the number of clusters when you run it with bias 0.99 on geometric shapes, you can get some neat results: At intermediate settings, it essentially interpolates between color quantization and construction of a centroidal Voronoi tesselation. With moderately high bias towards location, you get a tesselation with fuzzy edges that are sensitive to the original colors. With moderately high bias towards color, you get a somewhat location-sensitive color quantization. This is by clustering in all dimensions simultaneously, weighting some more heavily than others by an amount that depends upon the bias selected by the user. You might be onto something here, zarathoustra... although I don't intend to remove the ability to cluster in all dimensions, I'm not using the 0.0 or 1.0 ends of that bias slider right now... I could write some fast code based on small integers to perform strictly-color or strictly-location segmentation, and occupy the extreme ends of the slider with that. Or better yet, I could fork that stuff off into two different effects and put all of them in a Segmentation sub-menu... one based on color, one based on location. That would allow easy access to a couple of faster effects for what some people might think of as the most interesting uses, and I'd still be able to chip away on a "deluxe" version, which ideally is going to automatically select the bias. Probably by some simple method like standardizing the dimensions, which would give me a decent excuse to find/write some C# matrix inversion code. Anyone have any thoughts? Do you make use of the fine-grained control offered by the bias slider, or can I replace it with an automatic method? Do you ever use the palette creation option with the bias set to anything other than 0.01? Would you like to see a few specialized segmentation effects instead of a single catch-all effect?