Python Program Very Slow

[zaka100]

Sup, I've been designing a new image format, its called Compressed Network Graphics, Its supposed to be smaller than png (Its not ATM ). Its compressed using bz2 (Need help with this aswell, I need a better compression algorithm).

I'm using pygame to display the graphics and here is the code (very sorry about the mess):

import pygame
import bz2 as lzma
from pygame import gfxdraw
import sys
#from GetColour import *

inputfile = 'test.cng'

list2 = ['!','"','#','$','%','&',"'",'(',')','*','+','0','1','2','3','4','5','6','7','8','9',':',';','@','A','B','C','D','E','F','G','H','I','J','K','L','M','N','O','P','Q','R','S','T','U','V','W','X','Y','Z','[','`','a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z','~','?','/','{','}',']','|','─','│','┐','┘','├','┤','┬','┴','┼','═','║','╒','╓','╔','╕','╖','╗','╘','╙','╚','╛','╜','╝','╞','╟','╠','╡','╢','╣','╤','╥','╦','╧','╨','╩','╪','╫','╬','ª','À','Ø','Ë','ą','ü','ù','Ğ','ģ','ĵ','İ','Ķ','Ō','ʼn','ń','Ł','Ɗ','Ɔ','Ƈ','ƃ','Ƃ','Ɓ','DŽ','Dž','dž','LJ','Lj','lj','dz','Ǭ','ǧ','ǣ','Ǥ','Ƶ','ƶ','Ʒ','Ƹ','ƹ','ƺ','Ʀ','ɣ','ʋ','ɷ','ɶ','ɢ','ʊ','ʉ','ɵ','ɴ','ʈ','ʛ','ɠ','ʚ','ʙ','ɱ','ɝ','ɜ','ɛ','ɚ','ə','ɸ','ɤ','ʠ','ʡ','ʢ','ʣ','ʤ','ʥ','ʦ','ʧ','ʨ','Ḁ','ḁ','Ḃ','ḃ','Ḅ','ḅ','Ḇ','ḇ','Ḉ','ḉ','Ḋ','ḋ','Ḍ','ḍ','Ḏ','ḏ','Ḑ','Ḓ','ḓ','Ḕ','ḕ','Ḗ','ḗ','Ḙ','ḙ','Ḛ','ḛ','Ḝ','ḝ','Ḟ','ḟ','Ḡ','ḡ','Ḣ','ḣ','Ḥ','ḥ','Ḧ','ḧ','Ḩ','ḩ','Ḫ','ḫ','Ḭ','ḭ','Ḯ','ḯ','Ḱ','ḱ','Ḳ','ḳ','Ḵ','ḵ','Ḷ','ḷ','Ḹ','ḹ','Ḻ','ḻ','Ḽ','ḽ','Ḿ']
reserve = 'Ḿ'
listalpha = ['0','1','2','3','4','5','6','7','8','9','F']




dec = open(inputfile,'rb')
rtxt = dec.read()

#print(bytes(rtxt,'UTF-8'))


#compressed1 = bytes(rtxt,'UTF-8').decode('UTF-8')
#compressed = bytes(rtxt,'UTF-8')

decompress = lzma.decompress(rtxt)

'''
df = open('rawcngexample.txt','wb')
df.write(decompress)
df.close()
'''

lines = decompress.split(b'\n')



print(lines[0])



#print('lines[0]: '+bytes(lines[0],'UTF-8'))

res=lines[0].split(b'x')


width=int(res[0])
height=int(res[1])

screen_size = (width,height)

screen = pygame.display.set_mode(screen_size)
screen.fill(pygame.Color(255,255,255))
pygame.display.set_caption('Image Viewer')



'''
def makepix(x,y,color):
    screen.fill(color, ((x,y), (1, 1)))
makepix(10,10,(100,100,100))
'''

pygame.display.init()


def makepixel(x,y,color):
    #rgb_color = hex2color(color)
    #print(rgb_color)
    #screen.fill(rgb_color, ((x,y), (1, 1)))

    #gfxdraw.pixel(screen, x, y, color)
    #pygame.display.init()
    #pygame.display.update()
    pixels = pygame.surfarray.pixels3d(screen)
    pixels[x][y] = color

pygame.display.update()


#pixel(screen,(100,100,100),(100,100))
#pygame.draw.circle(screen, (0,0,0), (250,250), 125)

num=0
for i in lines:
    if num == 0:
        pass
    else:
        #print(bytes.decode(i))
        numy=0
        line = bytes.decode(i)
        n = 3
        splitinto3 = [line[i:i+n] for i in range(0, len(line), n)]
        for x1 in splitinto3:
            chars3=[]
            #x = chr(x1)
            #rgbs = getcolour(bytes.decode(chars3[0]),bytes.decode(chars3[1]),bytes.decode(chars3[2]))
            #rgbs = getcolour(chars3[0],chars3[1],chars3[2])
            numred = list2.index(x1[0])
            numgreen = list2.index(x1[1])
            numblue = list2.index(x1[2])
            
            #col = rgb_to_hex((numred,numgreen,numblue))
            makepixel(numy,num-1,color=(numred,numgreen,numblue))
            pygame.display.update()
                
            numy+=1
    num+=1
    #print('Line Complete')





#pygame.display.flip()
'''
def makepix(x,y,color):
    screen.fill(color, ((x,y), (1, 1)))
makepix(10,10,(100,100,100))
'''
pygame.display.update()


while True:
    for event in pygame.event.get():
        if event.type == event.QUIT:
            pygame.quit()
            sys.exit()

This is the part that is slowing the program down:
This:

def makepixel(x,y,color):
    #rgb_color = hex2color(color)
    #print(rgb_color)
    #screen.fill(rgb_color, ((x,y), (1, 1)))

    #gfxdraw.pixel(screen, x, y, color)
    #pygame.display.init()
    #pygame.display.update()
    pixels = pygame.surfarray.pixels3d(screen)
    pixels[x][y] = color

And This:

num=0
for i in lines:
    if num == 0:
        pass
    else:
        #print(bytes.decode(i))
        numy=0
        line = bytes.decode(i)
        n = 3
        splitinto3 = [line[i:i+n] for i in range(0, len(line), n)]
        for x1 in splitinto3:
            chars3=[]
            #x = chr(x1)
            #rgbs = getcolour(bytes.decode(chars3[0]),bytes.decode(chars3[1]),bytes.decode(chars3[2]))
            #rgbs = getcolour(chars3[0],chars3[1],chars3[2])
            numred = list2.index(x1[0])
            numgreen = list2.index(x1[1])
            numblue = list2.index(x1[2])
            
            #col = rgb_to_hex((numred,numgreen,numblue))
            makepixel(numy,num-1,color=(numred,numgreen,numblue))
            pygame.display.update()
                
            numy+=1
    num+=1
    #print('Line Complete')

And here is the link for the image file: https://www.mediafire.com/?me42i52w91rb9h2 (yep, 64kb for a 205x130 image)
Code download: https://www.mediafire.com/?hy8ryqq57d7b6e0

So, is there a way to speed this up other than using something like cython?


Edit: Please ignore the while loop at the end of the code

 

[carnageX]

Couple things that I could think is:
1) Compression of any kind is very taxing on the CPU - slower the CPU, the longer the compression will take

Is pygame an optimized library to be doing the sort of thing you're wanting to be doing?

Can you multithread your loops at all? That would help somewhat with time, if possible.

 

[Yami]

Neat project. You might wish to use Github for version control and hosting the code rather than Mediafire

Unfortunately I'm not a good enough programmer to actually help with your problem

 

[carnageX]

Neat project. You might wish to use Github for version control and hosting the code rather than Mediafire

+1. Or BitBucket if you want private code repo (allows private repos for free - whereas GitHub is paid for private repos).

 

[zaka100]

Couple things that I could think is:
1) Compression of any kind is very taxing on the CPU - slower the CPU, the longer the compression will take

Is pygame an optimized library to be doing the sort of thing you're wanting to be doing?

Can you multithread your loops at all? That would help somewhat with time, if possible.

Pygame to my knowledge is optimised for this sort of thing. I may try an opengl library and see if that speeds things up.

The compression and decompression speed is not a problem, it is actually the speed of reading all the decompressed data and displaying it.

I could try multithreading, I'd probably add it in a further update.

Neat project. You might wish to use Github for version control and hosting the code rather than Mediafire

Unfortunately I'm not a good enough programmer to actually help with your problem

True, I only added it on mediafire because so I could quickly upload it and post a link. As soon as I get it on GitHub or BitBucket, I'll post a link

 

[zaka100]

Added to github and partially fixed speed issue.

Link: https://github.com/zaka100/CNG-IMG