Source Printer Identification using Printer Specific Pooling of Letter Descriptors

Printed documents have been used traditionally for record-keeping, certification, and communication purposes. The digital revolution has propelled the use of digital documents in place of printed documents. However, many critical applications use printed documents like financial dealings, judicial process, administrative record keeping and communication, and certifications. The frameworks for many such applications require a combination of printed documents, scanned versions, and digital documents. This co-existence of printed and digital documents results from multiple reasons, including ease of use, operating cost, lack of digital expertise, and open digital security challenges. Technological developments have allowed the easy availability of good quality printers, scanners, and image-editing software. These are routinely misused by potential miscreants to create forged documents. The vast volume of printed documents and the evolving forging technology makes it challenging to analyze and detect forged documents and warrants the use of fast and accurate digital systems to analyze the authenticity of printed documents. The source printer’s information can provide important clues for forensic analysis of printed documents [1, 2, 3, 4].

Traditional methods use chemical techniques to investigate the chemical composition of toner spread on a paper. Chemical processes use spectroscopy [5] and x-ray [6] to discover a connection in seized documents. These methods require laboratory equipment and an expert to examine the samples. Also, these techniques require a significant amount of time and can damage the printed document in question. In sharp contrast, digital methods convert the printed documents into their digital counterpart using a reference scanner. Source printer identification with digital techniques has been gaining much attention. All the analysis is carried out in the digital domain; thus, it is faster and automatic. Two major digital approaches for source printer identification are discussed in the literature, namely, extrinsic (active) and intrinsic (passive) [1]. An extrinsic signature approach is an active approach in which a user embeds an extrinsic signal in the printed document before or during the printing process. However, they require active access to the printer, and a sophisticated signal embedding mechanism needs to be integrated with the printer [7]. Such methods are complicated and costly for the large volume of text documents printed by general-purpose consumer-grade printers. On the contrary, passive approaches rely on visually imperceptible printer artifacts induced intrinsically during the printing process [8].

In this paper, we work on source printer identification for text documents using an intrinsically induced printer signature. The existing methods process all the printed letters uniformly during the computation of the printer signature model for source printer identification in a close-set scenario (i.e., the user needs to know the set of all possible printers in advance). Our proposed method is based on the hypothesis that the printed letters exhibit location-specific variations due to the electrophotographic printing process’s characteristics [9]. We introduced the printer-specific local texture descriptor (PSLTD) in [4], a state-of-the-art handcrafted method for identifying the source printer using scanned images of printed documents. Most existing methods (including deep learning-based [2, 10]) do not perform well when the font type of letters in printed documents under test is not available in training. We term this as the cross-font scenario [4]. PSLTD outperforms all existing methods under a cross-font scenario.

We analyze the variation of PSLTD-based printer signature concerning the location of printed letters across the document. We show that the distribution of printer artifacts correlates strongly with printed letters’ location on the document (Figures 4). We introduce a printer-specific pooling technique that allows the prediction of source printer. We further improve the performance of PSLTD under a cross-font scenario using the proposed pooling technique. We show the efficacy of the proposed method using a correlation technique (Figure 2). The significant highlights of this work are as follows:

• 1.

  We provide an extensive analysis of printer signature variations across a printed document.

• 2.

  We introduce a location-based pooling technique that improves the performance of PSLTD for source printer identification.

• 3.

  We show the efficacy of the proposed pooling technique using a correlation-based prediction method instead of a complex classifier, thus paving classifier-independent prediction.

• 4.

  The proposed method performs better than state-of-the-art methods under the cross-font scenario achieving an average classification accuracy of 93.5%, 94.3%, and 60.3% on documents printed in Arial, Times New Roman, and Comic Sans font types when documents printed in only Cambria font are available for training.

The paper consists of the following sections. Section 2 briefly describes the related literature of intrinsic signature-based techniques for classifying the source printer of printed text documents. The details of our proposed pooling have been specified in Section 4. An extensive set of experiments are used to examine the effectiveness of the proposed method. The description and results of the proposed approach have been discussed in Section 5. Finally, we draw out conclusions from this work and the directions for future work in Section 6.

The problem of Source printer classification has gained much attention in the past decade [1]. This work is based on an intrinsic signature, so we focus only on them. Our method also extracts printer signature from images of printed letters, so we only discuss methods related to text documents. One of the early reliable printer signatures was based on the banding phenomenon, i.e., the appearance of light and dark lines perpendicular to the direction of the paper movement inside the printer [1]. It is a promising approach but requires documents images scanned at a very high resolution (2400 dpi).

From banding, the researchers moved towards texture-based methods. These methods are suitable for documents scanned at lower resolutions (300-600 dpi). The texture patterns created by the distribution of intensity values observed in the scanned document image form the basis for these methods. These variations are visually imperceptible to human eyes at commonly used font sizes. Nevertheless, their zoomed versions depict texture variations. These methods follow the traditional pattern recognition pipeline in which features are extracted from letter images followed by learning a suitable classifier model. The learned model is capable of predicting the source printer labels for each letter image. The majority voting over all the predicted letter labels provides the prediction for the whole document under test. The early methods in this category were developed for a specific letter type like ‘e’, which is the most frequently occurring letter in the English language. The features include gray-level co-occurrence matrices (GLCM) [11], GLCM extended in multiple directions, and scales [12], convolutional texture gradient filter (CTGF) based on filtering textures with a specific gradient [12], a combination of the discrete wavelet transform (DWT) and GLCM based features [13], a combination of features obtained after applying a spatial filter, Wiener filter, and Gabor filter [14], and GLTrP-based features [3]. A recent method proposes a decision-fusion model-based approach for source printer classification [15]. All these methods extract features that learn a model for a specific letter type except [3], which introduces a single-classifier approach that learns a single model for all letter types printed on a document. The use of all letters increases training samples, thus lowering the requirement on the number of training documents and also increasing predicted labels for a document under test to achieve good performance.

Elkasrawi and Shafait [16] and the authors in [10] proposed noise residual-based features. Kee and Farid [17] followed a different strategy and proposed a method based on the estimation of a printer profile. They create the profile using a mean character image (of letter type ‘e’), and top p eigenvectors (obtained by PCA) is generated for each printer. The mean character image is obtained from all occurrences of letter ‘e’ extracted by convolving each character image with a user-selected reference character image. Zhou et al. [18] proposed an interesting text-independent approach to identify source printer based on a piece of patented equipment to scan fine textures on printed pages. More recent work proposed a method for source printer identification using document images acquired using a reference smartphone camera [19].

Two methods have also been proposed using convolutional neural networks (CNN) [2, 10]. They replace the task of handcrafted feature extraction with a model learned by letter images. Both methods work using specific letter types (‘e’ and ‘a’). None of the techniques discussed so far have been evaluated for cross-font scenario (i.e., font type of documents in testing is not available during training.) This is crucial as there are so many font types available, and it is increasingly difficult to expect that all font types are available during training. A recent method based on printer-specific local texture descriptor (PSLTD) [4] tries to address this challenge. A completely different category is characterized by geometric distortion-based approaches [20, 21], which rely on features from translational and rotational distortions of printed text relative to its reference soft copy [22, 23, 24]. A detailed literature review has been covered in [12, 2, 3].

Printer specific local texture descriptor [4] is one of the most state-of-the-art digital systems for source identification of printed documents. The significant advantage of the PSLTD-based method is its performance under a cross-font scenario in which it outperforms existing methods by huge margins. PSLTD is a novel handcrafted feature extracted from each letter image capable of learning a single discriminative classifier model for all letter types. It belongs to the family of local binary patterns [25]. The design of our PSLTD emphasizes on preprocessing, thresholding and quantization, and encoding and regrouping stages of local binary feature extraction. It introduces a novel encoding and regrouping strategy built upon small linear structures of 3 $\times$ 1 shaped structures. The main hypothesis of PSLTD is that the distribution of such linear structures in small overlapping 3 $\times$ 3 patches can effectively encode and group local binary patterns. The method observes linear-shaped groups of pixels that exist around the center pixel at all possible orientations, i.e., horizontal (0^∘), vertical (90^∘), forward slant (45^∘), and backward slant (135^∘). Each 3 $\times$ 3 patch is assigned one or more line orientations based on intensity and gradient direction similarity. A pent-pattern vector is calculated from contiguous 3 $\times$ 3 sized patches, which is converted into five binary pattern vectors (BPVs). These BPVs from a local patch are mapped to multiple normalized pattern occurrence histograms based on uniformity [25] by encoding and regrouping them by their line orientations. The detailed mathematical description of PSLTD is beyond the scope of this paper. For more details, please refer [4]. The final PSLTD is a feature vector of 10502 dimensions comprising of four major components. The first group comprises normalized histograms obtained using intensity and gradient direction similarities as encoding and regrouping strategies. It is denoted by $\overrightarrow{F1}$ and is of 4425 dimensions. Similarly, the second and third groups are formed by considering only intensity similarity and only gradient direction similarity to encode and regroup the BPVs. They are denoted by $\overrightarrow{F2}$ and $\overrightarrow{F3}$ and are of 1475 and 4425 dimensions, respectively. The last component is formed by binary magnitude pattern vector (BMPV) denoted by $\overrightarrow{F}_{BMPV}$ and of 177 dimensions.

In this work, we analyze the change in PSLTD-based printer signature [4] concerning the location of printed letters across the document. Based on the analysis, we introduce a printer-specific pooling technique that enhances source printer classification accuracy. We try two types of pooling-topology, i.e., columns and grids. Our proposed system’s input is document images of hard copy printed documents acquired using a reference scanner. The choice of reference scanner does not impact the system performance as long as it is kept constant for all the documents. The proposed system comprises of four significant steps: 1) connected component extraction, 2) PSLTD-based feature extraction, 3) and printer-specific pooling, and 4) classification. The general pipeline remains the same as in [4]. The novel step is our proposed pooling strategy, which enhances classification performance. Figure 1 depicts an overall pipeline of the proposed system using classifier-based prediction. For images scanned and saved using 16-bit intensity values (our dataset [3]), the proposed method also allows prediction of source printer using a correlation technique instead of a complex classifier-based prediction. as shown in Figure 2.

We evaluate The performance of the proposed method on two datasets: a publicly available dataset [12], termed as DB1 from here on and another dataset [3] containing documents printed by 18 printers in four font types (termed as DB2). We do an extensive parameter search for both column and grid pooling to analyze printer signature variations. We chose DB2 as our baseline dataset for this purpose as it allows analysis under cross font scenario. The dataset DB2 is a higher precision dataset consisting of 16-bit depth intensity values compared to 8-bit depth in DB1. So, we analyze the impact of the correlation-based prediction technique on the DB2 dataset. The full-length PSLTD is a feature vector of 10502 dimensions [4]. We also compare the full-length feature vector’s performance and the approximated feature vector of a smaller length. In the remainder of this paper, we use PSLTD_4k and PSLTD_10k to denote the methods based on approximated smaller length feature vector and the original full-length feature vector, respectively. Also, we use the general term PSLTD has been used to denote PSLTD_4k to explain the experiments as we use PSLTD_4k for all experiments except in Section 5.3. The performance of our technique is compared with hand-crafted methods including GLCM [11], multi-directional GLCM (GLCM_MD) [12], multi-directional multi-scale GLCM (GLCM_MD_MS) [12], $\emph{CTGF-GLCM-MD-MS}_{e}$ [12] and CC-RS-LTrP-PoEP [3]. We also compare with data-driven methods of [2] denoted by CNN-$\{S^{raw},S^{med},S^{avg}\}_{a,e}$ and [10] denoted as CNN-$\{S^{raw},S^{nr}\}_{a,e}$.

We have proposed a new printer-specific pooling based system for source printer identification. We use an extensive set of visual analysis and experimental results to show that a printer’s signature varies across a printed document, and this variation can be captured by dividing the document image into non-overlapping blocks. We analyzed two variants for block pooling, i.e., column and grid pooling. For document images scanned at lower precision (i.e., 8-bit depth), we use SVM-based prediction (as used in most state-of-the-art methods). The proposed method performs better than most methods on the publicly available dataset and performs similar to the PSLTD-based method. Results show that column pooling performs better than grid pooling using an ensemble of SVMs as the number of samples available for each SVM (corresponding to each block) is insufficient to learn a discriminative model. However, for document images scanned at higher precision (i.e., 16-bit depth), the proposed method’s discriminative power is highlighted by a correlation-based prediction instead of a complex classifier. Both grid and column variants of the proposed method performs better than state-of-the-art methods when evaluated under cross font scenario. An extensive set of experiments reveal that the grid pooling variant performs better than column pooling using a correlation-based technique. The proposed method’s grid pooling variant achieves more than 93.5% and 94.3%accuracies when tested on documents containing Arial and Times New Roman font types and trained using documents containing only Cambria font type. Classification with Pearson correlation provides promising results that could pave the way for classifier independent detection approach with a high precision image acquisition process.

The proposed method and the existing state-of-the-art combination of PSLTD and consecutive pooling does not perform well on Comic Sans font as it is drastically different from the other three font types in our dataset DB2. The proposed method requires that the test document contains a sufficient number of letters printed in a block to estimate a good quality printer signature. The proposed method is less useful for intra-page forgery locations, as its localization ability is reduced if the prediction of source printer is carried out on fewer pooled feature vectors. There are other open challenges that need to be analyzed in detail, including toner variation, paper quality and type, and printer age. Future work will include improving the printer signature model to address cross-font type, cross-font size, and cross-language scenarios. It would allow the scaling of source printer identification systems to a larger number of printers.

This material is based upon work partially supported by the [http://www.serb.gov.in]Department of Science and Technology (DST), Government of India under the Award Number ECR/2015/000583 and supported by Visvesvaraya PhD Scheme, MeitY, Govt. of India MEITY-PHD-951. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the funding agencies.