Just a blog to share my tricks, code snippets

Saturday, December 23, 2017

Shamir's secret sharing scheme


Shamir's secret sharing is a threshold secret sharing scheme invented by Adi Shamir in his paper "How to share a secret" in 1979. In this scheme, a secret S is divided into n pieces in such a way that S is easily reconstructed from any k pieces (k < n) but even complete knowledge of k - 1 pieces reveals absolutely no information about S. Shamir secret sharing scheme is used in ad-hoc networks where there is no centralized infrastructure to distribute a Certificate Authority into several secured nodes in the network. This non-centralized approach have the advantages that there is no single point of security compromise and increase the virtual Certificate Authority's availability.
In this post I will explain how Shamir's secret sharing works and how to apply it into mobile ad-hoc network to form a virtual Certificate Authority.

Definition

The goal is to divide secret S into n pieces S1, S2, ... in such a way that:
  • Knowing any k or more Si pieces can easily reconstruct the secret S.
  • Knowing less than k pieces Si will not able to reconstruct the secret S.
This scheme is called (k, n ) threshold secret sharing scheme. if k = n, then every pieces of Si are required to reconstruct S.

Shamir secret sharing

The essential idea of Shamir secret sharing are from below comments:
  • Given f(x) = ax + b
    • Knowing more than 2 points on the graph of the function f(x) will reveal f(x) therefore reveal f(0)
    • Knowing less than 2 points on the group of the function f(x) will not reveal f(x) therefore not reveal f(0)
  • Given f(x) = ax2 + bx + c
    • Knowing more than 3 points on the graph of the function f(x) will reveal f(0)
    • Knowing less than 3 points on the group of the function f(x) will not reveal f(0)       
  • Given f(x) = ak-1xk-1 + … + a2x2 +  a1x + a0
    • Knowing more than k points on the graph of the function f(x) will reveal f(0)
    • Knowing less than k points on the group of the function f(x) will not reveal f(0)
From above comments, if S is f(0) we can divide S into n pieces by calculating n points on the graph of the function f(x). Each point is a sharing piece Si
Knowing more than k points will reveal S but knowing less than k point will not reveal S.

Example of Shamir's secret sharing

Suppose we have a secret S = 6 and want to divide the secret into 6 parts (n = 6) where any 3 parts can easily reconstruct the secret (k = 3)

Division process


Since the threshold is k = 3, we will generate a function f(x) = a0 + a1x + a2x2 which f(0) = S

Therefore, a0 = S and at random we obtain a1 = 166, a2 = 94

Our polynomial to produce secret shares (points) is therefore:

f(x) = 1234 + 166x + 94x2

In order to have 6 parts of the secret, we calculate 6 points on the graph of the function f(x).
D0 = (1, 1494)
D1 = (2, 1942)
D2 = (3, 2578)
D3 = (4, 3402)
D4 = (5, 4414)
D5 = (6, 5614)
Each point is a sharing secret

Reconstruction process

In order to reconstruct the secret any 3 points will be enough.
Let us consider (x0, y0) = (2, 1942), (x1, y1) = (4, 3402), (x2, y2) = (5, 4414)
We will compute Lagrange basic polynomials:
Secret S is the free coefficient, which means S = 1234

How to apply it to form a virtual Certificate Authority

By using Shamir’s secret sharing, the system secret or private key of the Certificate Authority is divided into n parts such that any k parts can perform as a Certificate Authority. Each part is given to a node in the network. The term server is used to refer to a node which keeps a part of a secret to participate in forming the virtual Certificate Authority. A server’s properties can be summarized as below:


  1. A server can be initialized securely with its share of the secret which allows it to act as a server.
  2. A server knows the public keys of all nodes in the network including ones will join the network after initiation.



Now consider an example when A need to find out the public key of B.
A sends out a broadcast message to its neighbors requesting a certificate for B.

  1. Each server which hears this message generates a partial certificate with its partial system
  2. secret and sends it to a combiner.
  3. A combiner combine partial certificates and generates a complete certificate and send to A. 
A combiner is just a server which takes on the responsibility of combining S partial certificates and generates a complete certificate. Any server can take on the role of a combiner. A server does not require any extra capabilities to be a combiner. Conversely, a server does not gain any extra information about the system secret by being a combiner

Thursday, November 9, 2017

An example of Github Rest API using Python


Suppose I want a python script to get some information of Customer IO's Github:
  • How many total open issues are there across all repositories?
  • Sort the repositories by date updated in descending order.
  • Which repository has the most watchers?

Github has an official REST API v3 document at HERE
Here are some notes before jumping into the script. They are all stated in Github API's document but I just want to make sure you are aware of:
  • For GET request, parameters are passed in the HTTP query string
curl "https://api.github.com/repos/customerio/esdb/subscribers?page=2"
In above example, the parameter page is passed in the query string. For PUT, POST, PATCH, DELETE requests, parameters are passed in the data of the request instead of query string. See the official document for more details
  • Requests that return multiple items will be paginated to 30 items by default. Information of pagination is in the response's header.
curl -i "https://api.github.com/repos/customerio/esdb/subscribers"
You will see the header will has a Link. next indicates the next page and last indicates the number of the last page
Link: <https: api.github.com="" page="2" repositories="" subscribers="">; rel="next", <https: api.github.com="" page="2" repositories="" subscribers="">; rel="last"       
  • You do not need to authenticate your requests. However, the rate limit for unauthenticated requests is only 60 requests per hour. Therefore, it is better to authenticate requests. For requests using Basic Authentication or OAuth, you can make up to 5000 requests per hour. First, you need to create an access token. Github has a guide at HERE
curl -H "Authorization: token OAUTH-TOKEN" https://api.github.com/repos/customerio/esdb/subscribers       

Now it is the full script

Tuesday, September 26, 2017

Free tools to record screen on mobile devices


While reporting bugs for mobile apps, you probably need to record the device's screen in order to show steps for reproducing the bug. Today I want to introduce 2 FREE tools that I'm using to record screen on iOS and Android devices.

iOS devices

I use LonelyScreen to record screen on iOS devices. LonelyScreen is a AirPlay receiver for Windows and MAC. It is like an Apple TV running on your desktop. You can cast anything from your iPhone or iPad to your computer screen just like a Apple TV.

Installation and Usage:

  • Download and install LonelyScreen on your PC at HERE
  • Turn on LonelyScreen on your PC
  • Connect wifi on your iOS device to the same LAN with your PC
  • Connect the right AirPlay Mornitoring on your iOS device. Then your iOS device screen should be visible on LonelyScreen app on your PC.
  • At the bottom of the Lonelyscreen app on your PC, there is a button to record. It will record your iOS screen with mp4 format

Note: 

  • It sometimes takes long time to connect your iOS devices to PC via AirPlay and ends with nothing shown on the PC. My solution is turning off and then on the wifi on your iOS device and re-connect the AirPlay Receiver
  • I also hit a problem with firewall on my desktop when connecting with the iOS device. Here is how to fix it http://as.lonelyscreen.com/firewall-solution.html . Basically, I added LonelyScreen in a list that allowed to go through firewall

Android devices 

Teamviewer is the best option.

Installation and Usage:

- Download and install Teamviewer on your PC at  HERE. It supports not noly Windows but also Mac and Linux
- Install Teamviewer Quicksupport on the Android device at HERE
- Open Teamviewer Quicksupport. It will show a ID.
- Open Teamviewer on your PC, input the ID above to connect to the Android device
- In order to record screen. Click to the button next to X one. Then click Files & Extras and then click Start session recording. It will save the record as a .tvs file. You can open tvs file with teamviewer. Teamviewer also provides a way to convert tvs file to avi file



Friday, September 22, 2017

Write a simple zip file password cracker with Python


In order to extract a zip file, Python provides a zipfile library which help us create, read write, append and list a zip file. However, It only supports ZipCrypto encryption. A complete guide about it is HERE

So the idea is:
- Read a file pw.txt that contains all passwords.
- Try to extract the file for each password
Here is the script:

Note: The zipfile library only support ZipCrypto encryption. That means when you use 7zip to archive a file, remember to use the ZipCrypto encryption method instead of AES-256 one

Thursday, August 31, 2017

Python: Some restrictions when using multiprocessing library on windows


I ran into one of these restrictions today and want to share it. Those restrictions are stated in multiprocessing library document at HERE . However, I want to explain them more and give some examples so we can understand them fully.

Restriction 1: Safe importing of main module


  • Problem:


Running the script below would fail with a RuntimeError

The error should be like this


  • Solution:


In the python document above, it states:
Make sure that the main module can be safely imported by a new Python interpreter without causing unintended side effects (such a starting a new process).
That means we need to put the all the commands inside a if __name__ == '__main__': block. That will prevent calling a new process (could be unintended) when import that module.
The code should be look like this

Restriction 2: More picklability

Restriction 3: Global variables

Details of o other restrictions will be updated later whenever I have a chance :)

Friday, June 30, 2017

Convert Windows's end of line characters into Unix, and vice verse, using Notepad++


The format of Windows and Unix text files differs slightly. DOS uses carriage return and line feed ("\r\n") as a line ending, which Unix uses just line feed ("\n"). That means when you write a bash script, for example, on windows and copy to to a Linux machine, executing it will result in errors.

  • Show end of line(EOL) characters: Go to View > Show Symbol > Show All Characters. In a unix text file, the end on line is the line feed("\n" or LF). In a windows text file, It shows ("\r\n" or CRLF)





  • To convert the a file from Windows to Unix text and vice verse: Go to Edit > EOL Conversion and then select the format you want.



  • Note: From the EOL Conversion menu, you can see the Mac also uses different end of line character ("\r" or CR)


Thursday, June 15, 2017

Enable SSH service for ESXi server



  • Login to the server using vShphere client
  • Select the host --> Configuration tab --> Security Profile --> Properties in the Services part



  • In the Services Properties dialog, select SSH and then click to Options



  • In SSH (TSM-SSH) Options dialog, select "Start and stop with host" and then press Start button



  • That is for starting SSH server. You may also need to start SSH client. Again, in the Security Profile, select Properties in the Firewall part



  • Select SSH client


Friday, June 2, 2017

Copy/paste between virtual remote console of vShphere client and the system


1. Login vShphere client and power off the VM
2. Right click the VM and select Edit Settings
3. Navigate to Options > Advanced > General and click Configuration Parameters.

4. Add 2 new rows as below:
isolation.tools.copy.disable FALSE
isolation.tools.paste.disable FALSE

5. Click OK and power on the VM

Appium Desktop has been released


Appium Desktop has been released for a while. This article is a brief introduction for this new app. Appium Desktop is a new open source app for Windows, Mac and Linux which is a GUI interface for using appium and inspecting your app's elements. Please be noticed that Appium Desktop is not the same as Appium. It is a combination of Appium with graphical interface and other tools.

Some cons of Appium Desktop:

  • Has GUI for Appium Server. You are able to set options, set ports, set path for logs,...Launching sessions is also possible without running code.
  • An inspector mechanism is added to inspect you app's structure without running other apps. You can also perform basic actions on it like tapping or sending keys.
  • Running faster than the old Appium GUI app. 
  • Able to work with Appium 1.6.0 or more.
  • No need to install via NPM. Installation is done just by downloading the exe/dmg/zip file.


Download: 
You can download it from this page. Currently, the latest one is 1.0.2-beta.2. Auto update is also provided whenever there is a newer version available.

Usage:

  • Start server: 

We can start a simple server or advanced option server.



  • Start session:

After starting the server, we will go to the server console output window which allows starting a new session or stopping the server. You can also get the server in this console windows instead of the log file which is configurable during starting server with advanced options.


  • New Session windows:

The New Session window allows you to construct a set of Appium desired capabilities used to launch an Appium session.

  • Inspector:

After the session is started, you are able to inspect your app's structure.

For more information about Appium Desktop, please visit its official github page. You can also check out this video


Thursday, June 1, 2017

How to show the size of all folders on Mac OSX


Finder list view only shows size of the files, not folders by default. Below trick will help show all the size of folders:

1. Open Finder and change to list view ( View > as List)
2. Right click on the any empty space in the finder view and select Show View Options.
3. Select Calculate all sizes in the dialog and then close it