Presidential Election 2019 Results – Analysis using Python

Election to select the 7th Executive President of Sri Lanka has been concluded recently. If you are interested in analyzing the results and seeking for data, following Python script would be useful to you. Please check the repository to get the code. Results are organized in Electoral District level. Sri Lanka is divided into 22 Electoral Districts and 2018 Registry of Electors were used in this election.

Code Repository : https://gitlab.com/bckurera/pres-election-19

Refer to the following example to understand how to use the module, prez_2019_ed_res

import prez_2019_ed_res as db

FORMAT_F_NUM = "5,.2f"
FORMAT_I_NUM = "9,d"

##Find out the district in which SLPP secured the highest votes (precentage/ count)##
def find_max_votes_per_party(p):
    obj = db.prez_2019_ed_res()

    for party in p:
        valid_votes = obj.get_column("Valid")
        votes_for_slpp = obj.get_column(party)

        temp_prec = 0
        temp_count = 0
        temp_prec_dist = ""
        temp_count_dist = ""

        for x in votes_for_slpp.keys(): 
            p = votes_for_slpp[x] / valid_votes[x]

            if(p > temp_prec):
                temp_prec = p
                temp_prec_dist = x

            if(votes_for_slpp[x] > temp_count):
                temp_count = votes_for_slpp[x]
                temp_count_dist = x

        print(party, " highest %     - ", format(temp_prec*100, FORMAT_F_NUM) , " % -", temp_prec_dist)
        print(party, " highest count - ", format(temp_count, FORMAT_I_NUM) , " votes -", temp_count_dist)

##Highest voters - turnout##
def voters_prec():
    obj = db.prez_2019_ed_res()

    valid_dist = obj.get_column("Valid")
    rejected_dist = obj.get_column("Rejected")

    res = {}

    for x in valid_dist.keys():
        p = rejected_dist [x] / (valid_dist[x] + rejected_dist [x])
        p = round(p, 4)
        ext = {p: x}
        res.update(ext)

    for x in sorted(res.keys(), reverse = True):
        print(" Rejected % - ", format((x*100), FORMAT_F_NUM), " - ", res[x], " count - ", format(rejected_dist[res[x]], FORMAT_I_NUM))

##Two Party world##
def two_party_analysis():
    obj = db.prez_2019_ed_res()

    slpp_votes = obj.get_column("SLPP")
    ndf_votes = obj.get_column("NDF")
    others_vote = obj.get_column("Others")

    res = []

    for x in slpp_votes.keys():
        if(slpp_votes[x] < (ndf_votes[x] + others_vote[x])):
            res.append(x)

    for x in range(0, len(res)):
        dist = res[x]
        combined = (ndf_votes[dist] + others_vote[dist])
        print("Combinely Secured - ", dist, " combined count - ", (combined), " % - ", format((slpp_votes[dist]/(combined))*100,FORMAT_F_NUM ), " %")


party = ["SLPP", "NDF"]
print("\n----------------------------------")
find_max_votes_per_party(party)

print("\n----------------------------------")
voters_prec()

print("\n----------------------------------")
two_party_analysis()

 

Working with Dictionaries – JSON in Python

In Python, Dictionary is a data structure in which data is indexed by the key value. Therefore in order to access it’s data members the key value is required. When converting data to JSON it is ideal to use dictionary as JSON format support the key-index and value pairing format. In this post, it is mentioned how to achieve dictionary to JSON and JSON to dictionary conversion.

Dictionary

A structure of a simple dictionary is as follow:

sample_dictionary = {'key-1' : 'value-1', 'key-2' : 'value-2' }

There is a function named dict( ) which can be used to create a dictionary.

JSON library

In Python, JSON library is readily available. Main two methods we are using are dumps and loads.

.dumps()

Converts a data structure in this case a dictionary to a JSON string.

.loads()

This method can be used to convert a JSON string to a dictionary data structure so that it can be accessed within the code using key-values of the indexes.

Study the following example, a dictionary is defined with few levels, termed as nested dictionary. Data of a Tennis match is arranged.

import json

match = {
	'match-name' : 'USA F35 Futures',
	'match-general-info' : 
		{'match-date' : '2018.12.04', 'match-time' : '22:30', 'match-place' : 'Tallahassee'},
	'player-A' : 
		{'name' : 'Random Name A', 'age' : 20, 
			'rankings' : {'local' : 4, 'ITF' : 17, 'results' : 
				{'winner': 35, 'runner-up': 20, 'break-down' :
					{'semi' : 10, 'quarter' : 4, 'R16' : 1}
				}
			}		 
		}
}


json_string = json.dumps(match,0)

print ('Data Set in JSON format \n' + json_string)

match_dict = json.loads(json_string)

no_of_semis = match_dict['player-A']['rankings']['results']['break-down']['semi']

print ('Number of semi-final matches the player played : ' + str(no_of_semis) + ' matches')

Note that how the values of the data structure are accessed using nested indexes. This method can be used to manage multi-level data hierarchy but it might be bit difficult to manage as the size of the data structure increases. Another disadvantage is to access the values, it is required to supply the key of the index.

Download files from a URL – Python script

Downloading files from a web server over the internet is a trivial task which is performed in our everyday life. Isnt it?

For example when accessing a web site, your browser downloads some files to your computer. Commonly, it could obviously be .html files and then it might include .css .js files as well. You dont notice it as it is handled and managed by the browser itself. However there can be instances that you need to access a specific URL and download some files. No doubt it can be done manually. What if there are hundreds of files to be downloaded. In this article it is discussed how to get it done with minimal effort using a Python script.

Disclaimer – Employing a script to access/ download content from a web server can cause a higher traffic towards the web server. Therefore do not use this knowledge to access any resource in a web server without getting prior permissions. Use this script at your own risk. The author bears no responsibility.

For the purpose we are going to use the library urllib which is an inbuilt library in Python. .urlopen and .read methods can be used to access the content of the resource and then .write method can be used to save the content to the local machine. Check how the code is organized.

Python Version 2.7.15

import urllib
url = 'http://url-for-resources'

webf = urllib.urlopen(url)
txt = webf.read()
f = open('local-machine-folder-path', 'w+')
f.write(txt)
f.close()

It is pretty straight forward. Then the next objective is to automate the process. Using a loop is the easiest. One issue to overcome though. How to get the file names. If the file names follow a regular sequence for example data_01.json, data_02.json, data_03.json etc, it is possible to generate the sequence. Otherwise file names should be inserted to the code. A solution could be to extract the file names to a file and then read the file and get filenames. 

Here is the Python script for the above approach.

import urllib

f = open("local-file-with-file-names", "r")
file_list = f.readlines()


for x in file_list:

	filename = x.strip()
	url = 'http://url-for-resources/'+ filename

	webf = urllib.urlopen(url)
	txt = webf.read()
	f = open('local-file-path-to-save/'+filename, 'w+')
	f.write(txt)
	f.close()
	
	print('Saving file ' + filename)

One line contains one file name in the file which is used to include files to be downloaded.

Enumerate in Python

Learning python is extremely easy though mastering it, is extremely difficult – by known

This post discusses the Enumerate function in python. It is a built-in function in Python.

enumerate (iterable, startIndex=0)

It takes an iterable object such as a list. The startIndex is a number by default it is 0 and denotes the starting index the function should return. The returned object is enumerate object.

Lets consider an example first.

Example

Print the values and indexes of the following list

list = [“Apple”, “Banana”, “Cherries”, “Dragon Fruit”, “Fig”]

Following python code will do the work:

fruit = ["Apple", "Banana", "Cherries", "Dragon Fruit", "Fig"]

for i in range(0, len(fruit)):
print(fruit[i] + ' is at index ' + str(i))

The same can be achieved with enumerate function as follow

fruit = ["Apple", "Banana", "Cherries", "Dragon Fruit", "Fig"]

for index, value in enumerate(fruit):
print(value + ' is at index ' + str(index))

Output of the script

$python main.py
Apple is at index 0
Banana is at index 1
Cherries is at index 2
Dragon Fruit is at index 3
Fig is at index 4

Hope you have understand how the function works. Indeed it is an easier and a cleaner way to select elements of a list.

Challenge

Can you try following:

The function should take a word and then the output should be something as in following pattern.

input – “Apple”
output – “A-Pp-Ppp-Llll-Eeeee”

input – “baNanA”
output – “B-Aa-Nnn-Aaaa-Nnnnn-Aaaaaa”

Simple Approach

def accum(s):

text = []

for i in range(0, len(s)):
getChar = s[i]
text.append(processChar(getChar, i))

return '-'.join(text)

def processChar(char, times):
text = char.upper()
for i in range(0, times):
text = text + char.lower()

return text

print(accum('Apple'))

The above code can be factored as follow, in just one line using enumerate

def accum(s):
return '-'.join(s[i].upper() + s[i].lower() * i for i in range(0, len(s)))

print(accum('Apple'))

However, it can be written as follow without using enumerate in one line.

def accum(s):
return '-'.join(t.upper() + t.lower() * i for i,t in enumerate(s))

print(accum('Apple'))

Even though it is possible to get the same output in different approaches, the code with enumerate looks clear, isnt it?

Happy 3.1415926 (π) day : 3-14

Happy 3.1415926 Day !

In every year, π day is celebrated on March 14th (3.14).

π is a super star in mathematics. It is mysterious and people keep exploring it. It is proven that this constant has no end hence it is irrational, that means it cannot be represented as a fraction. π is bigger, as big as the whole universe. In deed π is not the only irrational number we got. But π attracted many and we keep loving it.

Even though there is no known benefit of calculating the value of π people insist of doing it. For many engineering applications, having value of π for 3 or 4 decimal places would be sufficient. Other sophisticated scientific advanced might need much more accuracy but I am pretty sure, it is enough to know first 100 digits of π. But today, humans have calculated 22,459,157,718,361 digits[1] of π.

There are quite many activities are organized for the π day worldwide other than enjoying delicious pies. One of the interested things is ,contests on memorizing decimal places of π. The current world record is 70030 digits[2].

To celebrate the π day I have calculated π for 7 decimal places using Gregory and Leibniz series. It is one of the most primitive ways of calculating π though.

Happy 3.1415926 Day !

[1] https://en.wikipedia.org/wiki/Chronology_of_computation_of_%CF%80

[2] http://www.pi-world-ranking-list.com/index.php?page=lists&category=pi

PS: Following script gives 7 decimal digits of π.

ans = 0
for k in range (1,100000000):
      temp = ((-1)**(k+1)/(2*k-1))
      ans = ans + (temp*4)
print (ans)

 

Square and Multiply algorithm

Square and Multiply algorithm is an interesting algorithm which is also known as binary exponentiation algorithm as well. The algorithm is very useful against calculation of large integer powers of a number.

For an example, like calculating; (which is multiplying 3, four times), this is pretty straight forward:

In order to get the answer 3 rounds of calculations are required.

In that same manner, to calculate , 12 rounds of multiplication rounds are required. Nevertheless it is possible to use the following method and reduce number of required rounds.

Above is a short cut method to obtain the answer, however things might go weird when the exponent is really large, for example something like or .

Even using the shortcut method seems quite cumbersome but better than the first method.

However, in such situations Square and Multiply method comes handy. Even in the second method, we have applied Square and Multiply operation to get the answer. Using this algorithm make it easier to apply the logic behind, especially when it needs to be implemented as a computer program.

Taking back the earlier example, check the operations carried out:

(Square) 
(Square) 
(Square) 
(Multiply) 
(Multiply) 

what if the above is arranged as follow:

(Square) 
(Multiply) 
(Square) 
(Square) 
(Multiply) 

Even though that there is no much improvement in computational rounds, the second approach aligns with the Square and Multiple method. So as you might already have understood, the Square and Multiple method determines the combination of operations that needs to be carried out to reach to the final answer.

Algorithm

Following steps needs to be carried out :

1. Get the binary representation of the exponent.
2. Bits are read from left to right (MSB first) and it should start with a 1.
3. Starting value = 
4. Start scanning bits from the left.
5. As mentioned above the first bit must be 1.
6. If scanned bit is 1 then, square the value and then multiply by 
7. If scanneed bit is 0 then, square the value.
8. Repeat this for all the bits.

As an example, can be taken. Binary representation of 13 is: 

Since the first bit is 1, initially the value is squared and multiplied. Then next bit is taken, it is 1, therefore the value is square and multiplied again.  Then the third bit is 0, therefore the value is only squared. Final bit is 1, the value needs to be squared and multiplied. Pretty easy right !

Time to check this method for a larger exponent. Lets take following :

Steps:
(Square  ) 
(Multiply)
(Square )
(Multiply)
(Square )
(Square )
(Multiply)
(Square )
(Square )
(Square )
(Multiply)

Python Implementation

Following is the python code:

string bin(int num)

The above function takes an integer as the argument and then return the binary equivalent as a string. It starts with 0b prefix. Therefore it is required to consider [2:] of the string. As the first bit is always 1, the final output of the first step is always equal to x. That is the reason for starting the for loop from 3.

def exp_func(x, y):
    exp = bin(y)
    value = x

    for i in range(3, len(exp)):
        value = value * value
        if(exp[i:i+1]=='1'):
            value = value*x
    return value

Summary

Square and Multiply algorithm is a very useful algorithm which can be used to calculate values of integers having really large exponents. The number of calculation rounds is relatively less compared to the brute force method.

Greatest Common Divisor (GDC)

The Greatest Common Divisor (GCD) or Greatest Common Factor is the largest integer which is a common factor of  two or more integers. In other terms, GCD is the largest positive integer that divides each given integer.

For an example, GCD of 20 and 12 is 4. GCD of 18 and 12 is 6. Further, since primes dont have factors (other than 1 and the prime itself) the GCD of any two prime numbers is 1. There is no common factors for 20 and 11, therefore GCD for 20 and 11 is also 1. GCD is not limited to 2 integers, it can be for more than two. For an example, GCD of 30,60 and 90 is 30.

In order to calculate GCD of two numbers following algorithms can be used:

General Method

function cal_gcd(x, y)
Input: Let x and y are two integers

d = min(x,y)
while (True)
    if(x mod d is zero) and (y mod d is zero)
        return d
    else
        d = d - 1

Above algorithm can be used to calculate GCD of two numbers. With a slight modification, the code snippet can be changed so that it calculate GCD for more than two integers.

function cal_gcd_(list)
Input: list of integers
d = min (list)
while (True)
    if remainder = 0 for all integers in the list when divided by d
        d = d -1
    else
        return d

These two algorithms perform better, however once the integers are quite large, these algorithms are not efficient. For large integers, Euclid’s algorithm can be used. Python implementation of the above method mentioned below.

Python Code:

def gcd_gen_list(list_num):
    d = min(list_num)
    cal = True
    while(cal):
        counter = len(list_num)
        for i in range(0, len(list_num)):
            if (list_num[i] % d != 0) :
                d -= 1
                break
            else:
                counter -= 1
        if((d==1) or (counter == 0)):
            return d

Euclid’s algorithm

This is a pretty interesting method. This method is based on the divisibility concept.

Let, x is divided by y, so that the result is k and remainder is r. In such a case it can be expressed as;

x = k . y + r

Plus this method uses the following theorem.

GCD of integer x and y is equal to GCD of integer y and r.

Repeating this concept till remainder reaches zero.

Pseudocode algorithm:

function gcd_euc
Input: Let x and y are integers

calculate r (r = x mod y)
while (r is not 0)
   calculate r (r = y mod r)
return r

This method is pretty straight forward and very fast when compared to the method mentioned above for large integers.

Python code:

def gcd_euc(x, y):
    r = x%y
    while (r != 0):
        x = y
        y = r
        r = x%y
    return int(y)

Dijkstras Algorithm

This is another algorithm which can be used to compute GCD of two integers. It uses subtraction of numbers to calculate the GCD.

Pseudocode algorithm:

function gcd_dijk
Input: Let x and y are integers

while (x != y)
   If x > y Then
       x = x - y
   Else:
       y = y - x
return x

Python code:

def gcd_dijk(x, y):
    while(x!=y):
        if(x>y):
            x = x-y
        else:
            y = y-x
    return x

Binary method

The Binary method also known as Stein’s algorithm is also one of the algorithms to compute GCD. It is quite more complex than Dijkstras Algorithm. However unlike
Dijkstras Algorithm it uses division by 2, therefore it is possible to use bit operations.

Pseudocode algorithm:

function gcd_binary
Input: Let x and y are integers

while (x != y)
   factor = 1

   If x is 0 Then
       return factor * y
   If y is 0 Then
       return factor * x

   If x is even AND y is even Then
       factor = factor * 2
       x = x / 2
       y = y / 2

   If x is even AND y is not even Then
       x = x / 2

   If x is not even and y is even Then
       y = y / 2

   If x is not even AND y is not even Then
       If x > y Then
           x = (x - y ) /2
       Else
           y = (y - x) / 2
    
    return factor * x

Python code:

Python implementation is provided below. Note that bit-wise operator >> and << have been used instead of multiplication of 2 or division by 2.

def gcd_binary(x, y):

    factor = 1
    while(x!=y):

        if (x==0 or y==0):
            return factor*(x+y)

        if (is_even(x) == False and is_even(y) == False):
            if(x>y):
                x = (x-y) >> 1
                continue
            else:
                y = (y-x) >> 1
                continue

        if(is_even(x) and is_even(y)):
            x = x >> 1
            y = y >> 1
            factor = factor << 1 continue if(is_even(x)==True and is_even(y)==False): x = x >> 1
            continue
        else:
            y = y >> 1
            continue

    return factor * x

def is_even(x):
    y = x >>1
    if(y<<1==x):
        return True
    else:
        return False

All methods other than the General Method can be used to calculate GCD for 2 integers at once. In order to calculate GCD for more than two integers, it is possible to calculate GCD for pairs and then select the lowest GCD for all the pairs.

Python code:

def gcd_list(numList):
    numListSize = len(numList)
    GCDList = list()
    for i in range(0, numListSize):
        for j in range(0, numListSize):
            if(i!=j):
                ans = gcd_binary(numList[i], numList[j])
                GCDList.append(ans)
    return min(GCDList)

All methods can be found in bmaths library under bmaths_gcd.py
bmaths library is released under GNU General Public License v3.0

Local Authority Election 2018 – Parsing Results (Python Script)

The official results of the Local Authority Election 2018 just released via https://election.news.lk/ . If any one is looking for a script to parse the results, check the below link.

https://github.com/bckurera/elec_res_parse

The direct connection socket to the site has been removed. Therefore follow below steps unless you want to add your own socket to read the site (check my earlier post on Python Socket programming):

1. Open the relevant result page and save it in your local PC.
2. Rename the file name to 98.html (or change the file name in the script)
3. Execute the script.

This script has been released under GNU GENERAL PUBLIC LICENSE (Version 3)

Socket programming in Python

Python, in default offers low-level access to network services as many other languages do. This is often required when developing client-server applications or an application which needs to communicate with another service. Therefore in this article, it is discussed, how socket programming can be achieved in Python.

Low-Level Access

In this approach, python library socket is used. It is a part of the standard Python library, hence no manual installation is required.

Server code

This is a simple server which keeps listening to an incoming traffic from a client and then respond.

#!/usr/bin/python

import socket

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

HOST = '127.0.0.1'
PORT = 30303
BUFFER_SIZE = 2014

s.bind((HOST, PORT))

s.listen(5)
while True:
   print ('server running --')
   conn, addr = s.accept()
   data = conn.recv(BUFFER_SIZE)
   print ('Got connection from ', data, '-', addr)
   conn.send(b'Thank you for connecting')
   conn.close()

Note how the socket object has been initialized and then HOST address and the PORT have been defined. bind( ) method reserves the port for the host. accept( ) method accepts the incoming connection and then output another instance of the socket object as conn and the addr holds the address (and the port) of the incoming request.

When sending data using send( ) method, the content should be a byte-like object. No string objects can be used. That is why b literal is used. Or string.encode( ) can also be used here. The rest is pretty straight forward.

Client Code

Following is the code snippet for the client.
Code is almost as same as the server code other than not having a loop to listen to the incoming traffic. It simply send some data to the server and then receives the output from the server.

#!/usr/bin/python

import socket

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
HOST = '127.0.0.1'
PORT = 30303

s.connect((HOST, PORT))
s.send(b'username:user')
data = s.recv(1024)
print (data)
s.close ()

The server code needs to be executed first and then the client code.

However, since there is no threads in the server code, only 1 connection can be established with the server at a time.

Requesting a web page

It is possible to use the client code to contact a web server (or any other service) and request a web page. Or even contact a FTP service. The send request needs to be modified according to the protocol.

Following is a simple example on getting a web page using the client code. Localhost is used in the code however the HOST can be replaced with any URL which supports HTTP.

#!/usr/bin/python

import socket

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
HOST = '127.0.0.1'
PORT = 80

s.connect((HOST, PORT))
s.send(b'GET /test/index.php HTTP/1.1\n'
       b'Host: 127.0.0.1\n'
       b'Connection: keep-alive\n'
       b'Upgrade-Insecure-Requests: 1\n'
       b'\r\n')

data = s.recv(1024)
print (data)
s.close ()

Notice that only the send method has been changed to facilitate HTTP protocol. The output is a plain-text including header and body content. Further processing is required.

High-Level Access

For HTTP protocol, httplib module can be used too. However it is not a part of the standard Python library and it is required to get it installed manually. In the same manner, few non-standard libraries are available for FTP (ftplib), SMTP (smtplib), POP3 (poplib) protocols.

This is a simple introduction on socket programming in Python. According to your requirement the code needs to be modified. The intention was to demonstrate how Python can be used for the above mentioned purpose.

Move the last digit and double the number

Check if you can solve the following:

The objective is to find a number which gets doubled when the last digit of the number is moved to the front of the number.

For an example:

Let abcd is the number, it should get doubled when d which is the last digit, is moved to the front, forming dabc 

2 x abcd = dabc

can you find such a number or few numbers?

First I started building the following equation:

Let x is positive integer and y is a single digit.

x > 0 and 10 > y > 0

therefore, the number can be written as;

2 . (10 . x + y ) = (y . 10^p) + x 

Then,

x = [ (10^p) – 2 ] * [y / 19]

or

y = [19 . x ] / [(10^p) – 2 ]

Then after, it is possible to solve this using trail and error method. It is cumbersome, yet possible. I was tying to figure out a heuristic method to get the answer, but I could not. Therefore I though of using a python script to get the answer.

<pre>#!/usr/bin/python
def eqx(n, y):
    return int(((10**n)-(2))*(y/19))

def eqy(n, x):
    return int((19*x)/((10**n)-2))

def num_conv(num1, num2, p):
    d2 = num1%10
    d1 = int(num1/10)
    if(num2==((d2*10**p)+d1) and (10**p<num1)):
        return True
    return False

for i in range(0,300):
    for j in range(0,10):
        x = eqx(i,j)
        y = eqy(i,x)

        num1 = int((10*x)+y)
        num2 = int((y*(10**i))+x)

        LHS = 2*num1
        RHS = num2
        num_shuffled = num_conv(num1,num2,i)
        if ((LHS==RHS) and (num_shuffled) and (x!=0)):
            print('x,y,p',x,y,i)
            print (format(num1, ',d'),' shifting last digit ',format(num2, ',d'))
            print ('LHS',format(LHS, ',d'), ' RHS',format(RHS, ',d'))
            print('')</pre>

Output:

x,y,p 10526315789473684 2 17
105,263,157,894,736,842 shifting last digit 210,526,315,789,473,684
LHS 210,526,315,789,473,684 RHS 210,526,315,789,473,684

x,y,p 15789473684210526 3 17
157,894,736,842,105,263 shifting last digit 315,789,473,684,210,526
LHS 315,789,473,684,210,526 RHS 315,789,473,684,210,526

x,y,p 21052631578947368 4 17
210,526,315,789,473,684 shifting last digit 421,052,631,578,947,368
LHS 421,052,631,578,947,368 RHS 421,052,631,578,947,368

x,y,p 31578947368421052 6 17
315,789,473,684,210,526 shifting last digit 631,578,947,368,421,052
LHS 631,578,947,368,421,052 RHS 631,578,947,368,421,052

x,y,p 42105263157894736 8 17
421,052,631,578,947,368 shifting last digit 842,105,263,157,894,736
LHS 842,105,263,157,894,736 RHS 842,105,263,157,894,736

Later I found out these kind of numbers are called Parasitic numbers, OEIS sequence of A146088[2][3]. If you are interested, refer to the following Wikipedia article[1] as the starting point.

[1] https://en.wikipedia.org/wiki/Parasitic_number
[2] https://oeis.org/A146088/list
[3] https://oeis.org/A092697

This article is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported (CC BY-SA 3.0) license.