GSoC Weekly status update #8

I have changed the framework interface from its previous form, although the previous front end automated_rendering_testing.py is still present in the repo. Now the new interface, rendering_testing.py, need all the file names to be provided as command line arguments. The user gets the convenience  of using the tab completion this way. The user will have to give as command line arguments 6 files (font  file, test cases file, reference file, rendering output and files to store output) and an optional directory name(if the engine is harfbuzz).
 If the rendering engine is harfbuzz, user can run the script generate_hb_rendering.py  along with the test cases file and font file as parameters, to create the rendered output file. If that is not the case, the user will have to create this file as well in the prescribed form.

Now, the algorithm that actually test the rendering was a bit buggy and was giving certain wrong outputs for words with multiple rendering engines and I have cleared this error. This feature gives correct output now for the files I tried it with.
The next thing I am working on is the web interface and I am using Flask framework. Will make this code public as soon as I get the script running from the page.

GSoC Weekly update #6 & #7

The past two weeks has been a blur with a lot of travelling and minimal Internet access. The following are the works I have been doing so far:

The following modifications were asked to be made on the existing framework by my mentor after a Hangout session as part of the evaluations:
1. Modify the comparison algorithm so as to show positive results for the words with multiple correct renderings – This modification is made. Now, the user can give multiple glyph names separated by comma in the reference file and if the rendering matches any one of these, the framework will return a positive response.
2. Modify the reference glyph file, adding the glyph names of words with multiple correct renderings. Also some corrections were asked to be made in the existing reference file.
3. Modify the framework such that the user can even test by giving the file names as parameters. This one needs a little more work as I didn’t give options in argument parser for all the necessary file inputs. Will update this soon.
Along with these some minor fixes were asked to be done on the script and all those are taken care of.
As for the further developments, planned to create a web interface for this framework. I am trying to create this interface using Flask and I am currently working on it.
After that, the framework will be implemented in C. I have added a partially working implementation of this in the repo.
After the completion of all these, if time permits, references for other fonts are also planned to be made.
Will keep posted on further developments.
Thanks!

GSoC weekly update #5

The works this week has been a little slow with college exams and assignments. This is what I have done so far this week.

I have completed the list of reference files containing glyph names of 243 words from four fonts each. Fonts being: Rachana, Meera, Suruma and Lohit-Malayalaam.

The code has been modified to equip not only harfbuzz renderings but renderings from other engines line Uniscribe, provided the user will produce the output of the rendering engine herself/himself. I have created a Python package containing 2 modules each for testing and creating output. The main script automated_rendering_testing.py  will make use of this package to test and give the final result. To test the framework, one can just run ./automated_rendering_testing and then provide the necessary  information, when asked.

Coming to the tester, first it will compare the reference file and the rendering output. The it will create a file named result.txt containing the wrongly rendered words along with the number corresponding to the word in test cases’ file. This file is used only to create the png file of the wrongly rendered words, if the engine is harfbuzz. Other wise this file is ignored. Now the actual output is a file test_result.txt with the format:

Sl.No Word Rendering status(correct/wrong)

User can view this file, see the status and see the wrongly rendered word.

The framework works this way now: Image

And this would be the output.png file. (As I chose harfbuzz here)

Image

And the test_result.txt file would look like this:

Image

The agenda for this week is to re-write the whole code in C.

My code is available here: https://github.com/nandajavarma/Automated-Rendering-Testing

GSoC Weekly report #4

This week my main task was to migrate my code to Python. As of now I have implemented my algorithm in Python. Here is the link to the repo : https://github.com/nandajavarma/Automated-Rendering-Testing

I have expanded my test cases’ list a bit. Now it has 243 Malayalam words. I have manually created files with glyph names of these test cases in four fonts: Rachana, Meera, Suruma and Lohith-Malayalam in files names rachana-glyph.txt,  meera-glyph.txt etc. (It is still a bit buggy, so haven’t pushed the latest commit of this yet).

What the code basically does is, it will ask the tester which font she/he wants to test in. Say it is Meera. The code will look for the reference file which we manually create and the file with harfbuzz rendering of the test cases, names as hb_meera_rendering.txt. This file can be created by running harfbuzzrendering.py script with proper font files in the current directory. The main script rendering_testing.py will scan both these files and compare the glyph name corresponding to each word and stores the wrongly rendered words to a new list. Finally hb-view will be executed on the words inside this list and a file named output.png will be generated in the same directory that contains all the wrongly rendered words.

The baseline glyph names’ files aren’t ready yet with complete glyph names of all the 243 words. Will be able to complete it within 1-2 days.

GSoC Weekly update #3

This week I’ve been working on generating a baseline glyphs file for 4 fonts: Rachana, Meera, Suruma and Lohith-Malayalam. I have selected some malayalam words from harfbuzz tree and Santhosh Thottingal’s test cases which I thought would be enough to test rendering problems. Then I started listing the glyph names of these files for each fonts in separate text files. To get the corresponding Unicode code point of each word, I wrote a small Java code. So I executed the script on each word, found all the code points and made 4 text files that contains the corresponding glyph names of the four fonts I mentioned earlier. 

Although my mentor did tell me that it is not possible to generate glyph names automatically, I wasted more than a couple of days on a Font Forge script to make it automatically output the glyph names. But that gives the glyph name only if we click on each character, which became terribly disappointing. So instead I used it to make the baseline glyphs file in the structure I want if I click on the necessary characters. But this code is trivial as far as rendering testing is concerned and will leave it out from now (Just noting it down as it wasted a very non-trivial amount of my time 😉 ).

I have modified the main C code such that it will ask the tester which font she wants and after choosing the one she needs it will output the result based on the words I have given.

But my mentor pointed out that it looks quite messy looking at codes in 3 different languages for a single framework so I’ll be re-writing my code in Python this week.

You can find my code here: https://github.com/nandajavarma/Automated-Rendering-Testing (although the README is not up-to-date)

GSoC Weekly update #2

Coding period for GSoC has started the past week and I have been working on a very simple implementation of the proposal in C and two tiny bash scripts. My code is available here: https://github.com/nandajavarma/Automated-Rendering-Testing

The first thing to be done to test using these scripts is create a file that contains a set of words to be tested to see if their rendering is correct. Here I have taken a sample test data file created by SMC a while ago (ml-harfbuzz-testdata,txt). Now pass this file through the script render_test.sh along with the necessary font file. That is:

./render_test.sh ml-harfbuzz-testdata.txt /path/to/fontfile

This will create a file named rendered_glyphs.txt that contains the output of hb-shape function of harfbuzz, i.e. the glyph name followed by some additional numbers (which will be ignored for now).

Now create a file that contains the actual glyph names of the words in the the test data wordfile. I got the data from font forge. This has to be created manually and, as of now, obeying the following structure:

[glyph11,glyph12,glyph13,…,glyph1n]

[glyph21,glyph22,glyph33,….,glyph2n]

.

.

.

Also make sure that glyph names of each word is in the same order as that of the corresponding words in the test data file. I have named it orig_glyphs.txt Once this is done, we can pass the above two files through the executable of the script rendering_testing.c, say rendering_testing. That is:

./rendering_testing orig_glyphs.txt rendered_glyphs.txt

This script will compare the glyphs in order and if it find any pairs that doesn’t match, it will write to a file, result.txt, the line number in which the word appears in the test data file. Otherwise it will tell you the renderings are perfect.

Once this is done, to see the words with wrong renderings we will have to run the third script show_rendering.sh. It takes as input the result.txt file, the test data file and also the font file. That is:

./show_rendering.sh result.txt ml-harfbuzz-testdata.txt /path/to/fontfile

This script will create png images of the wrongly rendered words in the current directory.

That is all about my scripts. But the C code is very much inefficient. It even spits segmentation faults with some files. Once I make sure that I am on the right path after discussing with my mentor, I will be working on improving my algorithm and making this code better. That would be my next week’s work.