Simplifying the FlexiLayout by using an auxiliary element and a null hypothesis

This method can be used when you need to check the same bulky search constraints for several elements. The idea behind is that constraints are checked and the calculations are performed only once in some auxiliary element. As a result of checking the constraints, the given element is either searched for or not, so the found or unfound hypothesis is a marker signaling whether the constraints are met or not. This auxiliary element is created so that it can always detect something on the images, provided the DontFind() function (i.e. "stop the search") is not activated. The number of hypotheses shouldn't be too large, to avoid over extension of the tree of hypotheses and the time of the FlexiLayout matching. To achieve this, you can use, for instance, elements of the Object Collection or Paragraph types. These elements always generate a single hypothesis that includes all the objects of the specified type from the search area. In the Advanced pre-search relations section of the dummy element we write the set of conditions that we want to check for some of the elements located below the dummy element in the project tree. If all the conditions are met, we call the DontFind() function for the dummy element. In this case a null hypothesis will be generated for the dummy element, and it will serve as a marker that all the conditions have been met when the program starts searching for other elements. By doing so, we tell the program to run only the IsNull -check of the dummy element instead of checking the same bulky constraints for several elements.

Note.This method helps to make the code more descriptive. Additionally, if you need to edit the constraints, you can do this only in the description of the dummy element. It lowers the chances of logical and syntactical errors when duplicating the code.

Note.In the future versions of the product we plan to support creation of variables in the area of subelements of groups. The results of checking the constraints will then be stored in the values of different variables. The current method is a temporary solution (workaround) that helps to simplify the code in the Advanced sections in the present version of FlexiLayout Studio.

Let's see how this method works in the project 1.fsp (folder %public%\ABBYY\FlexiCapture\12.0\Samples\FLS\Tips and Tricks\Auxiliary element).

On these images, we will search for the following fields: "Invoice number", "Invoice date", "Company name", and "Company address".

Let's assume that we have to process invoices of two different types:

• The fields "Company name" and "Company address" are above the field "Invoice number";
• The fields "Company name" and "Company address" are below the field "Invoice number".

As you can see from the images, the fields "Company name" and "Company address" don't have names, which prevents us from using the standard procedure of looking for the date field by relying on its name. However, we can notice a certain pattern: when the date field is located to the right of the invoice number, the company name and address are located below the field "Invoice number" (pages 1 and 3); while if the date field is below the invoice number field, then the company details are above the field "Invoice number" (pages 2 and 4).

taking into account the detected pattern, it is would be best to search for the location of the fields "Invoice number" and "Invoice date" first. And then we are going to detect the other fields by relying on these two, specifying their mutual locations.

We have create a Group element InvoiceGroup that contains elements InvoiceHeader, InvoiceNum, DateHeader and a Group element DateGroup. These subelements are required to detect field names and the fields "Invoice Number" and "Invoice date".

Note.See Detecting dates in the case of low quality pre-recognition for more details on the methods of date search. Here we only describe the constraints for the date search in the current project.

As you can see, the date fields on the images do not always have names. So, when searching for a date field, you need to specify two sets of constraints: for cases when the name has been detected and for cases when the name has not been detected (a special case would be if the name is present on the page but has not been detected, for example, because of noise).

if not DateHeader.IsNull then
{ RightOf: DateHeader.Rect.Right;
Below: DateHeader.Rect.Top - 30dt;
Above: DateHeader.Rect.Bottom + 30dt;
}
else
{ RectArray ar;
Let ar1 = Rect (InvoiceNum.Rect.Right, InvoiceNum.Rect.Top-30dt, PageRect.Right, InvoiceNum.Rect.Bottom + 30dt);
Let ar2 = Rect (InvoiceHeader.Rect.Left, InvoiceHeader.Rect.Bottom, InvoiceHeader.Rect.Right + 300dt, InvoiceHeader.Rect.Bottom + 150dt);
ar = RectArray (ar1);
ar.Add (ar2);
RestrictSearchArea (ar);
}

If the name of the date field is detected (the constraint if not DateHeader.IsNull is checked), then the search will be carried out relative to the name of the date field: to the right of the name, on one horizontal level, with some margin of error for vertical displacement:

{ RightOf: DateHeader.Rect.Right;
Below: DateHeader.Rect.Top - 30dt;
Above: DateHeader.Rect.Bottom + 30dt;
}

Otherwise the search area is divided into two rectangles: to the right of the invoice number and on the same level as the invoice number and below the invoice field.

Note.For the sake of simplicity we assume that we have images of good quality where the field "Invoice number" and its name are always detected. In a real-life situation, before calling the properties of these elements we must run their IsNull-check, because if the elements are not detected, further search will be carried out relative to the search areas of the corresponding elements.

In the Advanced post-search relations section of the element Date we wrote the following code:

if (DateHeader.IsNull) and (not IsNull) then
{if (not InvoiceHeader.IsNull) then
{ FuzzyQuality: Rect.Left - InvoiceHeader.Rect.Right, {0, 0, 0, 50000}*dt;
FuzzyQuality: Rect.Top - InvoiceHeader.Rect.Bottom, {-50000, 0, 0, 50000}*dt;
}
if (not InvoiceNum.IsNull) then
{ FuzzyQuality: Rect.Left - InvoiceNum.Rect.Right, {0, 0, 0, 50000}*dt;
FuzzyQuality: Rect.Top - InvoiceNum.Rect.Bottom, {-50000, 0, 0, 50000}*dt;
}
}

It allows influencing the quality of the date hypothesis depending on its distance from the name of the invoice field and the field "Invoice number" itself if the name of the date field is not detected. The longer the distance to the specified fields, the greater the penalty will be for the corresponding hypotheses, i.e. we search for the date field nearest to the field "Invoice number".

Note.See Using Nearest and FuzzyQuality to look for elements for more details on the use of these functions.

We specify identical search constraints for the element DateAsString, but in the Advanced post-search relations section we add one more line to the above-mentioned code:

FuzzyQuality: 600dt - Width, {0, 0, 0, 50000}*dt;

This line is needed during date search, so that a hypothesis with a longer string of alphabet characters is preferable to a shorter one.

Additionally, on the Search Constraints tab of the element DateAsString we specified that, when searching for the date as a string of characters, the region of the element InvoiceNum must be excluded. This is because we decided, for the sake of simplicity, not to duplicate for the element DateAsString the same search constraints as we specified in the Advanced pre-search relations section of the element Date. We specified the search area as RestrictSearchArea (Date.Rect);. We thus told the program to search for the object of the element DateAsString in the area of the fuzzy rectangle of the element Date. The search area of the element Date can be represented as an array of rectangles. When a null hypothesis is generated for the element Date, the rectangle enclosing the search area is taken as a rectangle (Rect) of the current element. As you can see on the image below, it also encloses the field "Invoice number" described by the element InvoiceNum. Under some conditions (for instance, when the date field is very noisy), a situation may occur when, instead of the date field, the element DateAsString will detect the field of the invoice number, as characters (including digits) specified for this element do not have any format restrictions.

After the elements required to search for the fields "Invoice number" and "Invoice date" have been described, we can go on to the search for the fields "Company name" and "Company address".

We create an element of type Paragraph and name it ShamElement. This element serves as an auxiliary element and is used exclusively to check the mutual location of the fields "Invoice number" and "Invoice date".

In the Advanced pre-search relations section of the auxiliary element, we wrote the following code:

Let Date1 = InvoiceGroup.DateGroup.Date;
Let Date2 = InvoiceGroup.DateGroup.DateAsString;
Let DateGroup = InvoiceGroup.DateGroup;
Let InvoiceHeader = InvoiceGroup.InvoiceHeader;
Let InvoiceNumber = InvoiceGroup.InvoiceNum;

if ((not InvoiceHeader.IsNull) or (not InvoiceNumber.IsNull)) and
(
((Date1.IsNull == FALSE) and ((max (InvoiceHeader.Rect.YCenter - Date1.Rect.YCenter, Date1.Rect.YCenter - InvoiceHeader.Rect.YCenter ) < 30dt) or (max (InvoiceNumber.Rect.YCenter - Date1.Rect.YCenter, Date1.Rect.YCenter - InvoiceNumber.Rect.YCenter ) < 30dt)) and ((Date1.Rect.Left - InvoiceHeader.Rect.Right > 100dt) or (Date1.Rect.Left - InvoiceNumber.Rect.Right > 50dt)))
or
((Date2.IsNull == FALSE) and ((max (InvoiceHeader.Rect.YCenter - Date2.Rect.YCenter, Date2.Rect.YCenter - InvoiceHeader.Rect.YCenter ) < 30dt) or (max (InvoiceNumber.Rect.YCenter - Date2.Rect.YCenter, Date2.Rect.YCenter - InvoiceNumber.Rect.YCenter ) < 30dt)) and ((Date2.Rect.Left - InvoiceHeader.Rect.Right > 100dt) or (Date2.Rect.Left - InvoiceNumber.Rect.Right > 50dt)))
)
then
{ Dontfind(); }
else
{ Below: PageRect.Top; }

Despite the simplicity of constraints checked by this code, the code itself turned rather bulky. Its idea is that, if the date field detected by means of one of the elements Date or DateAsString is located to the right of the invoice number field but on the same level (with a vertical tolerance of 30 dt), then we tell the program that a null hypothesis must be generated for the auxiliary element. In the other cases the element ShamElement will be looked for below the top edge of the page. Since we created an auxiliary element of type Paragraph without any additional search constraints, it will enclose all text objects on the page and a single hypothesis will be generated.

Nearly all the constraints to be checked are intuitively clear. So we are only going to describe the most complicated of them.

max (InvoiceHeader.Rect.YCenter - Date1.Rect.YCenter, Date1.Rect.YCenter - InvoiceHeader.Rect.YCenter ) < 30dt

This part of the code checks that the date field (in this case, if it is detected by the element Date) is located on one horizontal level with the element of the name of the field "Invoice number". The date may be slightly higher or lower than the name (because of the possible errors during scanning or document filling). We specified that the maximum difference is 30 dt. We check the mutual vertical location of the field "Invoice date" and "Invoice number" by the same method.

Date1.Rect.Left - InvoiceHeader.Rect.Right > 100dt

This line checks that the date field is located to the right of the name of the field "Invoice number" (the coordinate of the left boundary of the date is greater than that of the right boundary of the name). We specified a tolerance of 100dt because between the name and the date there must be the invoice number itself. The location of the date field is to the right of the field "Invoice number".

All the conditions of the check are then duplicated for the element DateAsString.

To check that our code is correct, let's run the FlexiLayout matching procedure on all the pages. We see that on pages 1 and 3, where the date field is located to the right of the invoice number, a null hypothesis was generated for the element ShamElement. On the other two pages, text fragments were detected.

To detect the company details, we create a Group element CompanyGroup. It groups the elements CompanyName of type Character String (the company name on test images is written in a single line) and Address of type Paragraph. The element will be used to search for the block containing the company address.

The use of the auxiliary element helps to simplify the code which describe the search constraints of the element in the Advanced pre-search relations section. So, if the auxiliary element is not detected (i.e. a null hypothesis is generated for it), this will mean that the date field is located to the right of the invoice field. In such a case, the field "Company name" will be looked for below the field "Invoice number". If the auxiliary element is detected, the field "Company name" will be looked for above the field "Invoice number".

if ShamElement.IsNull then
{ Below: InvoiceGroup.InvoiceHeader;
Below: InvoiceGroup.InvoiceNum;
NearestY: PageRect.Top;
}
else
{ Above: InvoiceGroup.InvoiceHeader;
Above: InvoiceGroup.InvoiceNum;
}

Note.In this case we can use the function Nearest, as it helps to detect the field containing the company name unambiguously and accurately (there are no objects matching the same search constraints).

In the Advanced post-search relations section of the element CompanyName we write the following code.

if not IsNull then
{ FuzzyQuality: Rect.Left - PageRect.Left, {0, 0, 0, 50000}*dt;
FuzzyQuality: Rect.Top - PageRect.Top, {0, 0, 0, 50000}*dt;
FuzzyQuality: 100dt - Height, {0, 0, 0, 10000}*dt;
}

This code works when the single-line field "Company name" is looked for above the field of the invoice number. The sought fields don't have names. At the same time, the mutual location of the fields "Company Name" and "Company address" is different on pages 2 and 4. The constraint Above: InvoiceGroup.InvoiceHeader; and Above: InvoiceGroup.InvoiceNum; is met not only be the string with the company name, but also by the strings in the address field.

To select the only correct hypothesis among all the generated hypotheses, the following code in the Advanced post-search relations section is used.

By typing the code

FuzzyQuality: Rect.Left - PageRect.Left, {0, 0, 0, 50000}*dt;

and

FuzzyQuality: Rect.Top -PageRect.Top, {0, 0, 0, 50000}*dt;

we tell the program to stronger penalize hypotheses with objects close to the bottom and right boundaries of the image. However, on page 2 the field "Company name" is located to the left of the address field but lower then its upper line. So the described constraints are not sufficient to detect the sought field "Company name", and we have to add one more constraint.

FuzzyQuality: 100dt - Height, {0, 0, 0, 10000}*dt;

This line tells the program to check the height of all the lines for the generated hypotheses. The taller the characters in the line are, the higher the quality of the corresponding hypothesis.

After we have matched the FlexiLayout we see that the field "Company name" has been successfully detected on all the pages.

To detect the address field we wrote the following code in the Advanced pre-search relations section of the element Address:

if ShamElement.IsNull then
{ Below: CompanyName;
Above: TotalSumHeader.Rect.Top;
}
else
{
Above: InvoiceGroup.InvoiceHeader;
Above: InvoiceGroup.InvoiceNum;
RightOf: CompanyName.Rect.Left;
Exclude: CompanyName;
}

Here again we use the auxiliary element ShamElement, which helps to simplify the code.

The creation of a FlexiLayout is now complete. Once we have run the FlexiLayout matching procedure we will see that all the fields have been successfully detected.