Modelling and Estimation of Interior Orientation of Non-Metric Cameras using Artificial Intelligence

Abstract


Introduction
Non-metric digital cameras are not designed or manufactured with photogrammetry applications in mind and must be calibrated before any measuring. The interior orientation parameters can be calculated by calibration to treat the systemic error in the measurement process to reach the highest possible accuracy. Many programs and algorithms are designed for this purpose [1]. Camera calibration is an essential part of 3D modelling [2]. tried to look into in-depth all the black box processes that are going on in the software. Traditional calibration methods necessitate high-precision equipment and complex procedures. Calibration needs time and a professional person to do this. In some cases, using images taken by volunteers or old images by cameras with unknown interior orientation parameters saves time, cost, and effort in the remote monitoring process. To overcome this issue, some methods and techniques in artificial intelligence have been developed; these techniques provide estimated interior orientation parameters used in image correction without pre-calibration prosses. The literature on artificial intelligence in photogrammetry has highlighted several artificial intelligence studies [3]. A concise introduction to the actual mechanics of simulated annealing is provided [4]. The mathematical foundation of bundle adjustment (collinearity model) and self-calibrating [5]. The technique only requires the digital camera to observe a planar calibration target (checkboard pattern) displayed in at least two different view orientations from multiple perspectives. The camera or the planar calibration targets may be moved freely. The motion is unnecessary to know Modeling radial lens distortion. The camera's interior and exterior orientation parameters are retrieved using a closed-form method [6]. This paper explains the fundamental ideas underlying the numerous self-calibration approaches and discusses the principles underlying most self-calibration algorithms [7]. This study proposes a revolutionary auto-calibration technique that minimizes labor. Backpropagation and gradient descent is utilized to learn the network's parameters [8]. This research presents a new approach to camera self-calibration by merging EXIF data from digital camera images with an existing QPSO algorithm [9]. This research resulted in a better knowledge of how uncertainty propagates across the stereophotogrammetry system and a list of critical influencing elements based on the accuracy and precision of its photogrammetric results in acquiring geometrical measurements, and this study evaluated the potential of smartphone and digital cameras as data collection tools [10]. (i.e., surface area and volume). Propose an approach for automatically calibrating cameras based on the observations of pedestrians, and it suggested that they can estimate calibration parameters for scenes with several ground. Test results demonstrated the method's resistance to noise and outliers [11]. Propose a hybrid framework for camera calibration that blends learning-based strategies with conventional techniques to address these obstacles. In particular, this system employs CNN to accomplish distortion correction efficiently [12]. Several techniques have been developed for the calibration process. The calibration process is employed to estimate and model the interior orientation parameters to prepare these cameras for photogrammetric use. In the second section, an explanation of self-calibration and calculating IOPs for cameras also calculates the range value of these parameters. It also explains an approach to estimating IOPs by capturing two overlapping images for stereo calibration toolboxes (e.g., MATLAB toolbox and OpenCV) and the methodologies for checking the correlation between the actual IOPs and predicted. The third section will introduce the experimental results and the analysis of statistics resulting from a technique. Finally, the paper will conclude with a summary and recommendations for future work.

Methodology
For this purpose, smartphone cameras are chosen because they are widespread and easy for volunteers to deal with in this type of work (capturing an image for any feature and sending it to the researcher). Three types of smartphone cameras that differ in geometric properties and manufacturing quality were chosen. The first step is to assess the consistency of IOPs to know the upper and the lower limits values of the IOPs for the selected camera models. Consistency is evaluated by determining the difference in IOPs of digital cameras in several devices of each type and for the most popular smartphone model by performing self-calibration for several devices to calculate the IOPs and identifying the range of IOPs values and the limits of becoming known according to results values by performing self-calibration for sample size (30 devices). Principal distance, principal point, and lens distortion are included radial distortion coefficients (k1, k2) and tangential distortion coefficients (p1, p2). Second, establish a test board with a minimum of six Control Points. These many targets must be present in the images of each participant for stereo calibration. Programming an artificial intelligence system to estimate IOPs based on (the stereo calibration) function in the OpenCV software was used in the Python coding environment between two cameras making a stereo pair of images by generating random sets of parameters within the range that is calculated in evaluating the consistency step; these sets were substituted in (stereo calibrate) in OpenCV in the python environment, each set produces RMS re-projection error, and the optimal set of IOPs produces the minimum value of RMS re-projection error, six 3D control points appear in each image as shown in Fig 1. finally, the last step in that prosses is to evaluate Validation and Accuracy Comparison for results.

Self-Calibration
Self-calibration is a method for calibrating a camera that calculates the IOPs and fundamental projective geometry of a sequence of pictures. For this research, 30 non-metric digital camera devices of the same model (Test sample size) were used to increase the reliability of the data and perform statistical operations on the results. A Redmi Note2, TCL T700X, Sony Xperia Z1 smartphone cameras were chosen, as shown in Table 1. for the specification of these models. The first step was based on 12 photos for each camera to get accurate results.

Stereo calibration
Stereo Calibration is usually done on a unique design test field with at least six known adjusted control points Fig. 2. The two cameras' positions to observe this test field. The values of object Points and image points for the two images and the IOPs parameters are implemented as input information to computing the extrinsic parameters and the RMS re-projection error estimated for each pattern view.

Heuristics Simulated Annealing
Heuristics is an artificial intelligence technique used when optimization is difficult. Simulated annealing is a type of heuristics that ignores memory and uses random searching. A simulated Annealing is an approach to minimize RMS re-projection error by optimizing IOPs. All parameter values are created within (mean standard deviation) restrictions, utilizing (random) library python language. This Artificial Intelligence technique adjusts IOPs in a stereo calibration algorithm to calculate RMS re-projection for each trial, even if the initial RMS reprojection value was lower. It will cover all potential values and avoid becoming tied to the Local Minimum, as shown in Fig. 3. until it finds the optimal value (global minimum) after a predefined number of attempts to discover the best IOPs. Simulated annealing is an optimization algorithm to minimize the RMS re-projection error outcomes by updating the IOPs in every optimization process, as shown in Fig. 4, published in GitHub [13].

Results Analysis and validation
Simple statistical analysis was performed on the experimental results to compute the mean and standard deviation to compute the Relative Percent Difference% For Each Experimental Group, as shown in Table 2. And compute the coefficient of determination (R² or r-squared) or correlation coefficient to determine the variance between the IOPs values from traditional calibration and the predicted parameter's value. In other words, The R 2 coefficient of determination quantifies how well the regression predictions match the actual data; this function exists in the excel program. The statistical results are that shown in Table 3.

Results and Discussion
Three types of smartphone cameras were used to get experimental data for self-calibration. Thirty random devices evaluated each type of smartphone camera, and the IOPs results for each digital camera were represented in Table 2.
This system was evaluated by comparing the estimated value with the actual value calculated through pre-calibration by the coefficient of determination R 2, where these data and results were represented in Table 3. From Fig. 5. of the focal length estimation results and their comparison with the pre-calibration process-calculated results, three camera models for smartphone devices, ten for each model, were determined. Observed that the coefficient of determination R 2 for all devices is high and acceptable, which means a high correlation rate, particularly for the Sony Xperia device, where it exceeds 80 percent, and to a lesser extent for the TCL device. The Redmi Note 2 has a lower interconnection rate than the other devices in this test. In other words, R 2 increases with increasing the quality of the camera. In general, the correlation between the calculated and estimated focal lengths is high, and this high correlation is because the standard deviation is relatively small.  Xperia TCL As for the radial distortion coefficients for all types of devices, in Fig. 8. And Fig. 9. the coefficient of determination R 2 is extremely low, as there is no correlation between the results estimated by artificial intelligence and the results calculated by pre-calibration; consequently, it cannot be relied on and is deemed unacceptable. The failure of this system to estimate the radial distortion coefficients is attributable to the fact that these cameras were not designed for photogrammetry and image measurement, as they have a very large standard deviation for these coefficients, making it difficult to estimate the correct value due to this high degree of uncertainty.  Fig. 11 displays the outcomes of the tangential distortion, showing that the correlation ratio is quite weak between the estimated and calculated parameters. Thus, it is also not dependable as in the radial distortion coefficients.

Conclusion
This paper has introduced the system to estimate camera interior orientation parameters for images captured by unknown cameras using artificial intelligence technology and the estimated parameters as an alternative to camera parameters resulting from camera pre-calibration. The findings indicate that the IOPs strength of a relationship between the calculating and estimating by the proposed system is relatively high in principal distance and principal point with low lens distortion parameters because the parameters had high relative differences over the same model, which is significant due to cameras and lens manufacturing and cameras' daily use. Overall, this study strengthens the idea of using any image from any source in close-range photogrammetry applications. Further study must consider increasing the survey size to include more camera types with a larger sample size for each type.