2nd part of the project – testing at the function level (part 2)

August 14, 2010 at 9:18 am | Posted in Uncategorized | Leave a comment

Thanks to my mentor I have a list of functions that Hugin is using. It allows me to focus on the most important part of the project – writing the tests. And here is the list if someone needs it:

I also resigned from using Boost Test Library simply because ctest is already handling the most of BTL tasks (it also needs to install Boost library etc.). Not every element (function) from the list above is needed to be tested, but much of them is very important. So we concentrated on two essencial areas: math/projection and optimization. A list of tested functions (they are already added into svn and integrated with cmake/ctest):

– Optimizer related functions:
* CheckParams
* distSquared
* RunLMOptimizer (this is the main function)

– Math/projection related functions:
* transmercator_erect
* erect_transmercator
* stereographic_erect
* erect_stereographic
* sinusoidal_erect
* erect_sinusoidal
* rect_erect
* erect_rect
* pano_erect
* erect_pano
* mercator_erect
* erect_mercator
* sphere_tp_erect
* erect_sphere_tp
* sphere_tp_pano
* pano_sphere_tp
* sphere_tp_rect
* rect_sphere_tp


2nd part of the project – testing at the function level

August 2, 2010 at 12:31 am | Posted in Uncategorized | Leave a comment

I must say it was very hard for me at the beginning to determine what functions are more important and what are less but over time I started to recognize that. I will use the Boost Test Library for unit testing, for organizing tests CTest form CMake would be better because the first part of this project was based on it. Why Boost Test Library?

  • it is rather mature and stable
  • it has many information and examples of its usage
  • it has all the options needed for automatic testing
  • it has many additional options which might be used in future tests development
  • almost everyone have boost library already in their computer, so no additional installations are needed

more info about BTL : http://www.boost.org/doc/libs/1_34_0/libs/test/doc/index.html

Now some more about the tests. Here is a list I made myself for marking the priority of the testing files in libpano (the more ‘+’ the more it is important to test):

  • adjust.c                +++
  • bmp.c                    +
  • ColourBrightness.c+++
  • correct.c            ++
  • dump.c                +
  • fftn.c                +
  • file.c                ++
  • filter.c                ++++
  • fourier.c            +
  • hdrfile.c            +
  • javastub.c            +
  • jpeg.c                +
  • jpegicc.c            +
  • lmdif.c                +
  • math.c                ++
  • metadata.c            +++
  • morpher.c            +
  • multilayer.c        +
  • optimize.c            +++
  • pan.c                    +
  • PaniniGeneral.c    +
  • parser.c                +
  • perspect.c            +
  • png.c                    +
  • ppm.c                    +
  • PTcommon.c            ++++
  • PTDialogs.c            +
  • pteditor.c            +
  • ptfeather.c            +++
  • ptpicker.c            ++
  • ptstitch.c            +++
  • queryfeature.c        ++
  • remap.c                ++
  • resample.c            ++
  • rgbe.c                +
  • seamer_.c            +
  • seamer.c                +
  • shell_mac.c            +
  • sys_ansi.c            +
  • sys_common.c        +
  • sys_mac.c            +
  • sys_win.c            +
  • sys_X11.c            +
  • tiff.c                +++
  • Triangulate.c        +
  • ZComb.c                +

That is quite a big list but it is almost impossible to test every function from this files fully. So it is important to choice wisely. And now a bit smaller list with functions I will test in the next days (or already started testing):

  • void ColourBrightness(  fullPath *fullPathImages, fullPath *outFullPathImages, int counterImages, int indexReferenceImage, int parm3,int createCurvesType)
  • void CorrectImageColourBrigthness(Image *image, magnolia_struct *magnolia, int parm3)
  • void filter( TrformStr *TrPtr, flfn func, flfn16 func16, void* params, int color)
  • int panoCroppingMain(int argc,char *argv[], int operation, char *version, char *usage, char *defaultPrefix)
  • int CropImage(Image *im, PTRect *r)
  • int readJPEG ( Image *im, fullPath *sfile )
  • int writeJPEG ( Image *im, fullPath *sfile,     int quality, int progressive )
  • int readTIFF ( Image *im, fullPath *sfile )
  • int writeTIFF ( Image *im, fullPath *fname)
  • int writeHDR ( Image *im, fullPath *sfile )
  • int readHDR ( Image *im, fullPath *sfile )
  • int blendImages ( fullPath *f0,  fullPath *f1, fullPath *result, double s )
  • int panoTiffDisplayInfo(char *fileName)
  • int panoCreatePanorama(fullPath ptrImageFileNames[], int counterImageFiles, fullPath * panoFileName, fullPath * scriptFileName)
  • void  RunLMOptimizer( OptInfo    *o)
  • int panoFlattenTIFF(fullPath *fullPathImages, int counterImageFiles, fullPath *outputFileName, int removeOriginals);
  • int panoPSDCreate(fullPath * fullPathImages, int numberImages, fullPath * outputFileName, pano_flattening_parms *flatteningParms)

This list isn’t full and might be changed over time.

Finishing 1st part of the project

July 22, 2010 at 2:28 am | Posted in Uncategorized | Leave a comment

Checking and a lot of debugging resulted in some code changes.So a lot of a new reference files were added. Also, right now testing framework gained new ability to test the PToptimizer. I also struggled with a new found weird bug (TIFFOpen), this bug is not deterministic and if come out it just causing that one of the image isn’t created, and so the whole test fails. Some additional small changes might be needed in reference files (especially for parser). I also started to determining what functionality (what functions) need to be tested in function level testing framework and started designing this network (however it is still on very early phase). I will show some results of this issue soon.

Shortly main changes in the project:

  • perl script for PToptimizer testing
  • added reference files for PTroller, PTcrop, PTuncrop, PTtiff2psd and PToptimizer
  • removed and/or added new and/or modified reference files for most of the tools
  • some additional changes in test generation
  • other changes like comments etc.

PTroller, PTcrop, PTuncrop and PTtiff2psd tests

July 6, 2010 at 9:20 pm | Posted in Uncategorized | Leave a comment

This week I spent on creating new features to the project. Mainly I created some code to test some tools such as:

  • PTroller – right now it creates tiff files itself (similar to ‘simple’ tests used for each of input files), but it is possible to use as input files tiffs that are added to the project as reference
  • PTcrop – tests are constructed as in PTroller, but it is also possible to crop and then uncrop given image and check if the image is the same (comparing to the image which was taken to the processing)
  • PTuncrop – as above
  • PTtiff2psd – for comparing I used a new function (psdCmp) based on identify command -line tool from ImageMagick (more info about using this tool: http://www.imagemagick.org/Usage/basics/#identify and http://www.imagemagick.org/Usage/files/#info – use with -verbose to get more details about the given file).  I intended to check every layer, but when I was using ImageMagick I get results similar to this: http://www.rubblewebs.co.uk/imagemagick/psd.php, so right now only 1st method is used.

other changes made in the project:

  • fixed problems with parameter testing (new reference files needed)
  • small changes in compare.pl (mainly adjustments to cmake)
  • changes in cmake files (for adding tests)
  • other smaller changes in other files

Right now I’m working on extend this tools testing capabilities, creating and planing tests for other tools (especially for PToptimizer and PTmasker).

Last week report

June 29, 2010 at 11:24 pm | Posted in Uncategorized | Leave a comment

Thanks to Daniel M. German, the project gain new ability to dump the input data structures. Till now the main problem in adding new tests cases was the space they take. Now this problem is almost gone. Of course this method isn’t testing everything but in many many cases it is almost everything that is substantial in test case. The main ‘old’ tests must remain to test the most important elements of the project, new method should be essential part of testing that gives more information about problems/bugs that may be found (because much more test cases can be done that way). Now the main problem is the time that tests take, but this is a result of number of them – e.g. combination of projection, formats and interpolators gives about 20x10x23 = 4600 in worst case (when all options are checked) – but there is a solution: not every test must be executed, we can divide tests into two groups first that tests most important elements of the project and second that contains predefined sets of tests cases that tests specific part of the project. This could be done by customizing cmake (ctest), more info: http://www.cmake.org/Wiki/CMake_Testing_With_CTest#Running_Individual_Tests

most important changes in the project during last week:
in parser.pl
– new simple tool for testing with parser – parser.pl (basic version)
– added possibility for generating combined tests
– combine all parameters functions into one
– reference files will be extended if result won’t be found
– improved documentation and added some comments
– code improvements
– other less important changes
in other files
– added 64 new tests into cmake (16 are removed due to time they take)
– added reference files for new tests

Currently I’m working at adjusting existing tests with cmake and making some new tests to other tools.

A report that I should post week ago

June 29, 2010 at 9:29 pm | Posted in Uncategorized | Leave a comment

Here is a little summary of the project based on doc/stitch.txt file.
p – line options
w various widths are used during tests
h various heights are used during tests
f almost all projection formats are tested
P optional parameters for some projections – only defaults are used
v horizontal field of view of panorama – tested
n most important Panorama file formats are tested (only for f1 as input)
u not tested (default:10)
k not tested
b not tested
d not tested

o – line options
f f0 f1 f2 f3 are tested
v horizontal field of view of image
y various yaw angle are given in input images
p various pitch angle are given in input images
r various roll angle are given in input images
a,b,c lens correction coefficients are tested (for f0 and f2)
d,e initial lens offset are tested (for f0 and f2)
g,t initial lens shear are tested (for f0 and f2)
Tx,Ty,Tz,Ts not tested
+buf, -buf obsolete, not tested
S Selection, f2 has input images with S option, (no separate test for
this option)
C Crop (no separate test for this option)
m,mx,my,s obsolete, not tested

m – line options
g gamma, not tested
i interpolators, all are tested (separate from cmake)
f Invoke Faster Transformation, not tested

Changes that I made to the project in this two weeks:
– preparation for parser testing (I made some functions in code more modular, so
it would be easier to use and combine them)
– improved documentation and added some comments
– removed interpolators test from cmake testing process
– parameters tests don’t need anymore separate files (now they are generated
– added functionality to add custom tests
– code improvements
– other less important changes

Project status update.

June 2, 2010 at 10:49 pm | Posted in Uncategorized | Leave a comment

Main changes:
-removed old unused references
-added multiple changes in test generating script such as testing formats,
testing interpolators and antialising, simple testing, testing parameters
-added new references
-added additional logging procedures
-integrated testing process with cmake
-created runTest.pl for replace similar script written in python
-changes in comparing methods
-many other small changes
-and of course removed the dependency of enblend and changed loaction of the
Currently I’m working at cropping tests, code improvements and making more
comments and documentation to the project.

1st report

May 21, 2010 at 10:07 am | Posted in Uncategorized | Leave a comment

The things I done during this week:
+ created reference and tests for a circular fisheye lens
+ created reference and tests for a rectilinear lens
+ made changes in gen.pl (E0 R0 problem is now resolved,
unnecessary lines are now erased, and some other changes)
+ made changes in resizer.pl and resizer2.pl
+ mirror changes in panoAutomateTest.py
+ created simple README.TXT file
+ created new however still primitive tool for comparing images
“compare.pl” (it includes some different methods of comparison)
+ made some changes for a fullframe fisheye lens tests
+ made some changes for a equirectangular lens tests
+ created a new branch and add all files to the SVN repository
+ gathering information about adding regression tests using cmake

Plans for the next week(s):
– add additional parameters for some output projections in gen.pl
– add tests for other parameters
– add tests for cropping
– improve compare.pl
– log tool for tests
– set a rules that decide if the test is passed
– gather more information about adding regression tests using cmake

– so far reference images took already much space, but I have some
ideas to reduce it, however all of these ideas have flaws, the main
three ideas are:
1) use only one image for each format in a test – the result of enblend
use so the only results of enblend would be tested
2) use one format of image (for example only *jpg or *.tif) in a test and
ignore small difference between test and reference files
3) combining 1) and 2)

Hello world!

May 21, 2010 at 2:02 am | Posted in Uncategorized | 1 Comment

Blog still in update process. This is a test post.

Blog at WordPress.com.
Entries and comments feeds.