ISO/IEC JTC 1/SC 29/WG 1 N 238 
							    	     Date:   July 31, 1995

ISO/IEC JTC 1/SC 29/WG 1
(ITU-T SG8)

Coding of Still Pictures

JBIG                              JPEG
Joint Bi-level Image                     Joint Photographic 
 Experts Group                            Experts Group

TITLE:		Image Quality Test Result for JTC 1.29.10

SOURCE:		F. ONO (JBIG Rapporteur)

PROJECT:		JTC 1.29.10

STATUS:     		

REQUESTED
ACTION:		For Information

DISTRIBUTION:	WG1  Mailing List

Contact:
ISO/IEC JTC 1/SC 29/WG 1 Convenor - Eric Hamilton
C-Cube Microsystems, 1778 McCarthy Blvd., Milpitas CA  95035, USA
Tel:  +1 408 944 6335,   Fax:  +1 408 944 6314,   E-mail:  eric@c-cube.com  

ISO
INTERNATIONAL ORGANIZATION FOR STANDARDIZATION
ORGANISATION INTERNATIONAL DE NORMALISATION	
ISO/IEC JTC1/SC29/WG1
Coding of Still Pictures 

                                        					July  31 , 1995
 				                                  ISO/IEC JTC1/SC29/WG1/N238

Title:		Image Quality Test Result for JTC 1.29.10
Source: 	F.ONO (JBIG Rapporteur)

1. Image Quality test procedure
In evaluating picture quality of the proposed methods for JTC 1.29.10, JBIG had discussion in Epernay meeting, based on WG1 N213. The tseting procedure was as follows;
a) Subjective Test
Five-Point Scoring methods were adopted. The correponding judgement is as follows.

Score 	Judgement
5	Excellent
4	Good
3	Fair
2	Poor
1	Unacceptable
0	Failure
For text type images, following five methods were chosen to be evaluated.
#1. Proposal A (SPM)
#2. Proposal B (CPC)
#3. Original
#4. JBIG reduction (1/4)
#5. JBIG reduction (1/16)
For half-tone type images, following four methods were chosen to be evaluated.
#2. Proposal B (CPC)
#3. Original
#4. JBIG reduction (1/4)
#5. JBIG reduction (1/16)
b)Legibility Test
For legibility testing, following four methods were chosen to be evaluated.
#1. Proposal A (SPM)
#2. Proposal B (CPC)
#3. Original
#4. JBIG reduction (1/4)
For legibility testing, we had Image f10_200 and Image f10_300, produced by scanning the same original image with two different resolutions. By checking the reproduced image quality of four methods, Point 3 & Point 4 letters were chosen from 300 dpi data (Image f10_300), and Point 4 & Point 6 letters were chosen from 200 dpi data. One test data set contains forty letters of small point and forty letters of large point. Since we have four methods and two resolutions, totally eight test set was produced. The arrangement of letters in each test set was randomized to avoid the guess of reviewers from preceding test sets.

2. Test Result
The comparison of coding Bit-rate for each method is provided in Table1. It should be noted that bit-rate is the number of coded bit being normalized by that of JBIG lossless coding (single progression sequential). The 5-point subjective picture quality for each method is provided in Table 2. The legibility test result is provided in Table 3.
The Bit-rate vs Quality relation is given from Fig.1 to Fig.4. From Fig.1, it can be said that both SPM and CPC have superiority to current standard for text images. From Fig.2, CPC’s superiority to current standard can be observed, but not so much as its superiority for text images.
From Fig.3 & Fig.4, we cannot introduce the superiority of both new methods from the current standard, or which of the new two is better, because the curve is supposed to be not so smooth (usually the slope with clamping for both ends), and we cannot fully control the bit-rate of new methods and current methods.
The CPC method got higher quality point than Lossless (original image) for subjective test. However, since the deviation is somewhat large, we cannot assert that it is significant from statistic sence.

Table.1 Bit-Rate of each method for test images to be used

TEXT-TYPE BitRate
		LOSSLESS	Lossy
		JBIG  	SPM	SPM 	CPC 	JBIG(1/4) JBIG(1/16)
f01_200 	 1.0	0.7232	0.4308	0.3536	0.4877	0.2262
f02_200  	 1.0	1.0558	0.9023	0.9842	0.4929	0.2434
f03_200 	 1.0	0.7991	0.5037	0.4638	0.4763	0.1952
f04_200  	 1.0	0.6036	0.3291	0.2098	0.4548	0.1924
f05_200 	 1.0	0.7378	0.4983	0.6933	0.4586	0.1992
f06_200  	 1.0	1.0020	0.7649	0.9076	0.4532	0.1955
f07_200  	 1.0	0.7428	0.4983	0.4088	0.4503	0.1805
f08_200  	 1.0	1.0449	0.9119	1.0277	0.4811	0.2435
f05_300  	 1.0	0.7308	0.4247	0.3450	0.4787	0.2149
f05_400  	 1.0	0.7453	0.4390	0.3487	0.4761	0.2195
f05_600  	 1.0	0.8123	0.4653	0.4232	0.4482	0.2117
f07_300  	 1.0	0.7022	0.4478	0.3274	0.4706	0.2055
f07_400  	 1.0	0.6976	0.4360	0.3118	0.4812	0.2191
f07_600  	 1.0	0.8231	0.4910	0.5112	0.4586	0.2127
Average	 	 1.0	0.8014	0.5346	0.4986	0.4692	0.2114

HALF-TONE-TYPE Bit Rate
		LOSSLESS	Lossy
		JBIG  SPM	SPM 	CPC 	JBIG(1/4) JBIG(1/16)
f11_400	1.0	-     	-     	0.7891	0.2383	0.0887
f12_400	1.0	-	-	0.9508	0.3039	0.1538
f13_400	1.0	-	-	0.7667	0.0834	0.0423
f14_400	1.0	-	-	0.6921	0.4457	0.1764
f15_600  	1.0	-	-	1.0538	0.4690	0.1693
f16_800 	1.0	-	-	0.7234	0.3118	0.1465
Average  	1.0	-	-	0.8293	0.3089	0.1295

Legibility Bit Rate
	   	LOSSLESS	Lossy
		JBIG	 SPM	 SPM 	  CPC 	   JBIG(1/4) 
f10_200	1.0	0.9540  	 0.6761   0.7858    0.4267	
f10_300	1.0	0.8772	 0.6247   0.6762    0.4464	

Table.2 Subjective Quality Test Result ( 5-point Score)

	Text-type	Half-Tone	
SPM	4.213	-	
CPC	4.428	3.488	
Lossless	4.414	3.536	
JBIG(1/4)	2.849	1.895	
JBIG(1/16)	0.955	1.245	

Table. 3 Legibility Testing Result
(# of error characters per 40 letters)

	Lossless	SPM	CPC	JBIG(1/4)	
200dpi 4Point	 3.857	15.571	5.286	30.857	
200dpi 6Point	 0.714	 1.7143	1.000	10.571	
300dpi 3Point	 1.714	 8.2857	3.571	26.423	
300dpi 4Point	 0.857	 1.7143	1.571	 6.571	

Fig.1

Fig.2

Fig.3

Fig.4