moments of retardation View previous topic :: View next topic  


NFOHump.com Forum Index - The Useless Void
Page 1 of 1
Nui
VIP Member



Posts: 5720
Location: in a place with fluffy towels
PostPosted: Fri, 24th Feb 2017 18:40    Post subject: moments of retardation
There were some circumstances that could almost explain this...

Anyway, I just accidentally wrote a code path which reads a png image, decodes it, does nothing with it, encodes it to png and writes it down in a different place. I was done by the time I realized I was copying an image. Laughing

I blame python for making this too easy. And God. And Bob 'Asshole' Barnsen while we're at it.
Back to top
TSR69
Banned



Posts: 14962
Location: Republic of the Seven United Provinces
PostPosted: Fri, 24th Feb 2017 18:58    Post subject:
Lossless encoding?

Formerly known as iconized
Back to top
Bob Barnsen




Posts: 31973
Location: Germoney
PostPosted: Fri, 24th Feb 2017 19:02    Post subject:

Enthoo Evolv ATX TG // Asus Prime x370 // Ryzen 1700 // Gainward GTX 1080 // 16GB DDR4-3200
Back to top
Nui
VIP Member



Posts: 5720
Location: in a place with fluffy towels
PostPosted: Fri, 24th Feb 2017 19:03    Post subject:
TSR69 wrote:
Lossless encoding?

I think it is, but I don't think it redeems me Laughing (not does it matter in this case)
Back to top
paxsali
Banned



Posts: 18352
PostPosted: Fri, 24th Feb 2017 19:31    Post subject:
⁢⁢


Last edited by paxsali on Thu, 4th Jul 2024 21:16; edited 1 time in total
Back to top
Bob Barnsen




Posts: 31973
Location: Germoney
PostPosted: Fri, 24th Feb 2017 19:33    Post subject:
@paxsali
You think you are some cool dude programmer.
But in the end you are it not.

Enthoo Evolv ATX TG // Asus Prime x370 // Ryzen 1700 // Gainward GTX 1080 // 16GB DDR4-3200
Back to top
paxsali
Banned



Posts: 18352
PostPosted: Fri, 24th Feb 2017 19:35    Post subject:
⁢⁢


Last edited by paxsali on Thu, 4th Jul 2024 21:16; edited 1 time in total
Back to top
Nui
VIP Member



Posts: 5720
Location: in a place with fluffy towels
PostPosted: Fri, 24th Feb 2017 19:46    Post subject:
paxsali wrote:
Maybe show some code?

Well sure mate, here you go.
Code:
import numpy as np
from PIL import Image
def copy_image_increased_cpu_cycles(fn_in, fn_out):   
    img = Image.open(fn_in) # read image from disk, decode to 'Image'
    arr = np.array(img) # turn into numpy array
    img = Image.fromarray(arr) # turn into 'Image" again
    # encode to an image format based on file ending and write to disk
    img.save(fn_out)

With this function you can much less efficiently copy only images* (!) from one place to another. Use it wisely Laughing

* there is probably some exceptions to this for certain image formats. Please consult the Pillow libray (PIL) for that. Laughing

To my defense this is not how the code actually looked, I just condensed it to the stupid part
Back to top
paxsali
Banned



Posts: 18352
PostPosted: Fri, 24th Feb 2017 19:49    Post subject:
⁢⁢


Last edited by paxsali on Thu, 4th Jul 2024 21:16; edited 1 time in total
Back to top
Nui
VIP Member



Posts: 5720
Location: in a place with fluffy towels
PostPosted: Fri, 24th Feb 2017 19:56    Post subject:
paxsali wrote:
And I have an early prototype for video encoding/compression that uses a specific transformation algorithm to represent any video to one single 2D image in a way that it can easily be (whole-)compressed super-good (but with astronomical resource-hunger).

Do you have example statistics? Small video encoded to h.264/h.265 and your image encoding in terms of compression ratio, encoding/decoding speed and memory usage Very Happy


Actually I would be interested how you decoded videos for python. At some point, Maybe I wanna try again to teach a neural network to do motion compensated frame interpolation. Dataset generation is rather simple, because you can just get high frame rate youtube videos, and take triplets of sequential video frames for training (first and last as input, middle as teacher). Problem is... currently I can't accelerate any neural network framework because I have an AMD card. But hopefully tensorflow will support AMD as they want to. Tensorflow is awesome Smile
Back to top
Nui
VIP Member



Posts: 5720
Location: in a place with fluffy towels
PostPosted: Fri, 24th Feb 2017 19:59    Post subject:
paxsali wrote:
But does the first img differ at all from the img after you load it with fromarray?
If not, then you might aswell skip the whole steps, no?

Like just img = Image.open(fn_in) ... to make sure it can be loaded as Image with no exceptions,
then immediatelly img.save(fn_out), done. No?

Nothing of value happens there! Basically just some headers get created describing the data, and then switched for some other header and back again. The data should be completely untouched. Part of why this is so retarded.

edit: In fact, in the best case scenario this is what happens: PIL loads the data and creates some header which among other things points to that data. Then numpy creates another header which also points at the same data. To go full circle I create another header that can or should be the same as the first one that also points at the same data. Of course since I overwrite the name 'img' the first header will be marked for deletion Laughing

The other reason is that with
Code:
shutil.copyfile(src, dst)

you could just copy any kind of file straight Laughing


Last edited by Nui on Fri, 24th Feb 2017 20:05; edited 1 time in total
Back to top
paxsali
Banned



Posts: 18352
PostPosted: Fri, 24th Feb 2017 20:02    Post subject:
⁢⁢


Last edited by paxsali on Thu, 4th Jul 2024 21:16; edited 1 time in total
Back to top
Nui
VIP Member



Posts: 5720
Location: in a place with fluffy towels
PostPosted: Fri, 24th Feb 2017 20:09    Post subject:
I think I tried to use ffmpeg, but I only remember ending up with pngs for each frame and it was extremely slow!

Do you load the raw format in python? Did you even use python for this compression thing? Very Happy
Back to top
harry_theone




Posts: 11252
Location: The Land of Thread Reports
PostPosted: Fri, 24th Feb 2017 20:10    Post subject:
I don't like pythons.
Back to top
Nui
VIP Member



Posts: 5720
Location: in a place with fluffy towels
PostPosted: Fri, 24th Feb 2017 20:11    Post subject:
How about gophers? https://en.wikipedia.org/wiki/Go_(programming_language)

kogel mogel
Back to top
paxsali
Banned



Posts: 18352
PostPosted: Fri, 24th Feb 2017 20:18    Post subject:
⁢⁢


Last edited by paxsali on Thu, 4th Jul 2024 21:16; edited 1 time in total
Back to top
Nui
VIP Member



Posts: 5720
Location: in a place with fluffy towels
PostPosted: Fri, 24th Feb 2017 20:34    Post subject:
Ah okay. Of course such simple formats can just be read, d'uh. But they have some header at least right?

It was already clear from your first description, but essentially you coerce image encoders to use their spatial compression for temporal compression. Plus 25 frames are compressed as one.

Now first subdivide all frames into rectangular patches first, compress them individually. Now only pixels within a spatial window are within each image. Maybe if you go far enough with this you can end up with an insane method that is equivalent to normal video encoders Laughing

Would you care to show actual code for this zig-zag pattern? Could you post an image, or do they look so bad that you can't tell anything anymore?

Oh and because this is the UV: screw you for thinking in fucking PAL frequencies. 25 frames for a second, geez. Even 50Hz sucks. 60Hz is so much better. You can actually use BFI for motion blur reduction without insane flickering or image repetitions. Mad

kogel mogel
Back to top
Bob Barnsen




Posts: 31973
Location: Germoney
PostPosted: Fri, 24th Feb 2017 20:36    Post subject:
harry_theone wrote:
I don't like pythons.

What she said to Mandingo.

Enthoo Evolv ATX TG // Asus Prime x370 // Ryzen 1700 // Gainward GTX 1080 // 16GB DDR4-3200
Back to top
tonizito
VIP Member



Posts: 51487
Location: Portugal, the shithole of Europe.
PostPosted: Fri, 24th Feb 2017 20:41    Post subject:
@NuiBut can you do it in assembly? Smug
 Spoiler:
 

boundle (thoughts on cracking AITD) wrote:
i guess thouth if without a legit key the installation was rolling back we are all fucking then
Back to top
Nui
VIP Member



Posts: 5720
Location: in a place with fluffy towels
PostPosted: Fri, 24th Feb 2017 21:38    Post subject:
Laughing

Well. If we go with a high level emulation of my code above we could optimize it to a copy operation. I think we would just have to prepare filenames as parameters to a call to a system function for copying files.
So, encode two strings... put 'em on a stack... call os function somehow.... and don't I need some entry point?!? GAH NO NO NOO
Back to top
Guy_Incognito




Posts: 3436
PostPosted: Fri, 24th Feb 2017 22:08    Post subject:
Today I wrote (in C90) a prototype of shared library implementing multiple producers - single consumer scenario using threads, semaphores and mutexes. Builds and works on Linux and Windows from the same project. A shitload of #ifdefs.

Bob, am I better than Python programmers?
Back to top
paxsali
Banned



Posts: 18352
PostPosted: Fri, 24th Feb 2017 22:21    Post subject:
⁢⁢


Last edited by paxsali on Thu, 4th Jul 2024 21:16; edited 1 time in total
Back to top
Nui
VIP Member



Posts: 5720
Location: in a place with fluffy towels
PostPosted: Fri, 24th Feb 2017 22:36    Post subject:
paxsali wrote:
Code:
#       10        20        30        40        50        60        70        80
#--------|---------|---------|---------|---------|---------|---------|---------|

aiming to please the eye? Do you still do this? Very Happy

color me crazy but I think both versions may have the important part missing Very Happy
paxsali wrote:
Version A:
Code:
def transform_video():
   pass

Version B:
Code:
def transformed_video_from_file():
   pass


But you didn't just move the time axis to the first position right? In that case the transform could be done with np.rollaxis Razz
Back to top
paxsali
Banned



Posts: 18352
PostPosted: Fri, 24th Feb 2017 22:43    Post subject:
⁢⁢


Last edited by paxsali on Thu, 4th Jul 2024 21:16; edited 1 time in total
Back to top
Nui
VIP Member



Posts: 5720
Location: in a place with fluffy towels
PostPosted: Fri, 24th Feb 2017 23:19    Post subject:
Well thanks for the python code then Laughing

Maybe I should take a look at ST3, at least for home use. Here I use vim, but I'll never get used to it at this speed. At home mostly notepad++ or vim over ssh Razz


There is actually one useful program I wrote and it happens to be in python. But I started writing that when I basically just learned of python and it works as such. It's spaghetti code, doesn't use numpy, there is manual formatted file reading and writing.

Anyway, it uses argyllcms to read from a colorimeter (aka a monitor calibration device). I use it to calibrate displays. As far as I can tell, it actually has support for one interesting feature even CALMAN does not support, being white point adaptation. So I am happy about that Very Happy. I won't post that abomination though Laughing
Back to top
Bob Barnsen




Posts: 31973
Location: Germoney
PostPosted: Sat, 25th Feb 2017 13:05    Post subject:
Guy_Incognito wrote:
Today I wrote (in C90) a prototype of shared library implementing multiple producers - single consumer scenario using threads, semaphores and mutexes. Builds and works on Linux and Windows from the same project. A shitload of #ifdefs.

Bob, am I better than Python programmers?


Yes!

Enthoo Evolv ATX TG // Asus Prime x370 // Ryzen 1700 // Gainward GTX 1080 // 16GB DDR4-3200
Back to top
Guy_Incognito




Posts: 3436
PostPosted: Sat, 25th Feb 2017 14:35    Post subject:
Thanks!
Back to top
Page 1 of 1 All times are GMT + 1 Hour
NFOHump.com Forum Index - The Useless Void
Signature/Avatar nuking: none (can be changed in your profile)  


Display posts from previous:   

Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB 2.0.8 © 2001, 2002 phpBB Group