Sign in to follow this  
Ketenks

Memory is Low, It wont Load Two PDNs

Recommended Posts

1 minute ago, toe_head2001 said:

Your image has 20 layers. I haven't seen your image, but I know the layers in my images are mostly transparent with just a few object/elements. Lots of transparency equals lots of opportunity for compression.

Ah, now it makes sense. Why shouldn't there be a way to "compress" the layers in memory? What you are saying is that paint.net treats every layer as a whole new image, but what if it didn't. What if it only considered the graphics that were there and gave coordinate positions to each separate graphic? Then it wouldn't need to store or decompress all that transparency.

Share this post


Link to post
Share on other sites
11 minutes ago, Ketenks said:

Why shouldn't there be a way to "compress" the layers in memory?

 

Because paint.net needs to read what's in memory. Compressing and decompressing memory would just tax your CPU, and cause slight delays.

 

11 minutes ago, Ketenks said:

What you are saying is that paint.net treats every layer as a whole new image

 

Correct.

 

11 minutes ago, Ketenks said:

but what if it didn't. What if it only considered the graphics that were there and gave coordinate positions to each separate graphic? Then it wouldn't need to store or decompress all that transparency. 

 

Yes, that would be ideal. That's how Photoshop does it (if I remember correctly).

Edited by toe_head2001

Share this post


Link to post
Share on other sites

It would be interesting to learn why a 20 layer file decompresses into a file size that could contain more than an hour of video recorded in HD. Strange indeed!

Share this post


Link to post
Share on other sites

Video codecs get to "cheat" in this regard, as they don't have to encode the full image in every single frame. They just encode the differences compared to the previous frame.

Share this post


Link to post
Share on other sites
13 hours ago, Ketenks said:

Why shouldn't there be a way to "compress" the layers in memory? What you are saying is that paint.net treats every layer as a whole new image, but what if it didn't. What if it only considered the graphics that were there and gave coordinate positions to each separate graphic? Then it wouldn't need to store or decompress all that transparency.

 

There is a way, and I even know what to do and (mostly) how to do. It just hasn't been implemented yet, and it'll be a rather large project. Parts of this have already been completed with each major update to Paint.NET. For example, the version 4.0 release finished what was necessary for the "back end" of the rendering engine (4.0 was a 5 year project). Now the "front end" needs to be updated, which is where the layers are stored. Each layer is currently stored as 1 monolithic bitmap allocation, and the final work will be the break this into tiles. So for each (e.g.) 128x128 pixel region, if it's empty (or just homogenous -- same thing) then only a few bytes of memory will be needed.

 

Storage isn't even the difficult part. It's making sure all of the other parts of the various rendering systems are able to deal with this.

 

There's already quite a lot of work queued up for 2019, so don't expect this soon. However, as you can see in my blog post, development velocity is greatly accelerated now.

Share this post


Link to post
Share on other sites
14 hours ago, toe_head2001 said:

Video codecs get to "cheat" in this regard, as they don't have to encode the full image in every single frame. They just encode the differences compared to the previous frame.

I understand that part. So I need to clarify my point.

I just download a 3:11 minute UHD video in MP4 format from Vimeo. The compressed file size was 278 MB. 
There was enough variation between the content of the many scenes to make my point that a video ( recorded at ~30 fps x 191 sec = 5730 frames (layers)) that takes (on average) as much storage space as 20 frame image.

To state it more concisely: his 20 layered image takes up as much space on average as 1910 frames of UDH video.

Edited by HyReZ
to add more nfo and clarification

Share this post


Link to post
Share on other sites
8 hours ago, Rick Brewster said:

 

There is a way, and I even know what to do and (mostly) how to do. It just hasn't been implemented yet, and it'll be a rather large project. Parts of this have already been completed with each major update to Paint.NET. For example, the version 4.0 release finished what was necessary for the "back end" of the rendering engine (4.0 was a 5 year project). Now the "front end" needs to be updated, which is where the layers are stored. Each layer is currently stored as 1 monolithic bitmap allocation, and the final work will be the break this into tiles. So for each (e.g.) 128x128 pixel region, if it's empty (or just homogenous -- same thing) then only a few bytes of memory will be needed.

 

Storage isn't even the difficult part. It's making sure all of the other parts of the various rendering systems are able to deal with this.

 

There's already quite a lot of work queued up for 2019, so don't expect this soon. However, as you can see in my blog post, development velocity is greatly accelerated now.

That's awesome! I'm glad to hear it.

 

One thing though, isn't tiling the image different from rendering a collection of pixels that touch each other based on their top-left corner position? If the entire image is coded based on only what is there, then tiling would break up the continuity of the pixels that touch each other. You could connect sparse pieces together by drawing a direct line of transparency to every group of pixels separated by transparency and render the whole image as one group of pixels where only the pixels that have information are stored and the few lines of transparency that connect them and the coordinate position of the top-left most pixel with respect to the canvas.

 

That's what I thought would erase storing all the transparency information. I don't understand what the tiling represents, so I can't say what it means but I'm trying to make my "coordinate position" idea more clear since they seem to be doing different things in my mind.

Share this post


Link to post
Share on other sites
2 hours ago, HyReZ said:

I understand that part. So I need to clarify my point.

I just download a 3:11 minute UHD video in MP4 format from Vimeo. The compressed file size was 278 MB. 
There was enough variation between the content of the many scenes to make my point that a video ( recorded at ~30 fps x 191 sec = 5730 frames (layers)) that takes (on average) as much storage space as 20 frame image.

Compare compressed file to compressed file. Your compressed video was 278MB and my compressed image of 20 layers is 93MB. So your video is 3x the size of the image while it is compressed and if it was decompressed then I'd imagine they'd be a little closer to each other's value since the image file has a ton of transparency and the video has a regular distribution of variation throughout it's content.

Share this post


Link to post
Share on other sites
40 minutes ago, Ketenks said:

That's awesome! I'm glad to hear it.

 

One thing though, isn't tiling the image different from rendering a collection of pixels that touch each other based on their top-left corner position? If the entire image is coded based on only what is there, then tiling would break up the continuity of the pixels that touch each other. You could connect sparse pieces together by drawing a direct line of transparency to every group of pixels separated by transparency and render the whole image as one group of pixels where only the pixels that have information are stored and the few lines of transparency that connect them and the coordinate position of the top-left most pixel with respect to the canvas.

 

That's what I thought would erase storing all the transparency information. I don't understand what the tiling represents, so I can't say what it means but I'm trying to make my "coordinate position" idea more clear since they seem to be doing different things in my mind.

 

Nothing I said would change how the program behaves for you as the user. It would all be ... *wait for it* ... transparent.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this