On non-combinatorial weighted total least squares with inequality constraints

Observation systems known as errors-in-variables (EIV) models with model parameters estimated by total least squares (TLS) have been discussed for more than a century, though the terms EIV and TLS were coined much more recently. So far, it has only been shown that the inequality-constrained TLS (ICTLS) solution can be obtained by the combinatorial methods, assuming that the weight matrices of observations involved in the data vector and the data matrix are identity matrices. Although the previous works test all combinations of active sets or solution schemes in a clear way, some aspects have received little or no attention such as admissible weights, solution characteristics and numerical efficiency. Therefore, the aim of this study was to adjust the EIV model, subject to linear inequality constraints. In particular, (1) This work deals with a symmetrical positive-definite cofactor matrix that could otherwise be quite arbitrary. It also considers cross-correlations between cofactor matrices for the random coefficient matrix and the random observation vector. (2) From a theoretical perspective, we present first-order Karush–Kuhn–Tucker (KKT) necessary conditions and the second-order sufficient conditions of the inequality-constrained weighted TLS (ICWTLS) solution by analytical formulation. (3) From a numerical perspective, an active set method without combinatorial tests as well as a method based on sequential quadratic programming (SQP) is established. By way of applications, computational costs of the proposed algorithms are shown to be significantly lower than the currently existing ICTLS methods. It is also shown that the proposed methods can treat the ICWTLS problem in the case of more general weight matrices. Finally, we study the ICWTLS solution in terms of non-convex weighted TLS contours from a geometrical perspective.     

Total least squares adjustment in partial errors-in-variables models: algorithm and statistical analysis

The weighted total least squares (TLS) method has been developed to deal with observation equations, which are functions of both unknown parameters of interest and other measured data contaminated with random errors. Such an observation model is well known as an errors-in-variables (EIV) model and almost always solved as a nonlinear equality-constrained adjustment problem. We reformulate it as a nonlinear adjustment model without constraints and further extend it to a partial EIV model, in which not all the elements of the design matrix are random. As a result, the total number of unknowns in the normal equations has been significantly reduced. We derive a set of formulae for algorithmic implementation to numerically estimate the unknown model parameters. Since little statistical results about the TLS estimator in the case of finite samples are available, we investigate the statistical consequences of nonlinearity on the nonlinear TLS estimate, including the first order approximation of accuracy, nonlinear confidence region and bias of the nonlinear TLS estimate, and use the bias-corrected residuals to estimate the variance of unit weight.     

拟牛顿修正法解算不等式约束加权总体最小二乘问题

根据总体最小二乘准则,可以将附有不等式约束的变量误差（errors-in-variables,EIV）模型转化为标准最优化问题,并运用有效集法、序列二次规划法等优化方法求解。已有算法在涉及计算目标函数的Hesse矩阵（二阶导数）时,存在计算量较大的缺陷。针对上述问题,利用基于拟牛顿法修正Hesse矩阵的序列二次规划算法解算附有不等式约束加权总体最小二乘问题,新算法减小了计算量,可以提高收敛速度。通过实例,证明了该算法具有很好的适用性和计算效率。     

二维直角建筑物规则化的加权总体最小二乘平差方法

针对基于遥感数据的二维建筑物的直角化问题,以建筑物边界点的坐标为观测值,以顾及边界正交限制条件的直线斜率和截距为参数,建立附有限制条件的变量误差（errors-in-variables,EIV）模型。考虑观测向量和设计矩阵相关的情况,给出了增广设计矩阵的协方差阵的计算方法,推导了附限制条件的通用加权总体最小二乘（weighted total least squares,WTLS）平差算法,以及近似精度评定算法和仅含二次型限制条件的WTLS平差方法。理论和算例分析表明,在建筑物重建问题中,附有限制条件的EIV模型比经典附有限制条件的Gauss-Helmert模型易于构建,所提的WTLS算法快速收敛速度快,对拓展WTLS平差方法的应用具有理论与实践意义。     

多变量稳健总体最小二乘平差方法

分析指出了在总体最小二乘解下,含有多列独立变量的(以下简称为多变量)变量含误差(errors-invariables,EIV)模型,其各列变量的改正数受对应的参数估值与观测向量先验精度的联合影响,参数估值与观测向量先验精度的乘积越大,则该列变量的改正数越大。因此,现有稳健总体最小二乘方法采用同一个单位权中误差对多变量EIV模型进行降权处理时,会优先对模型中的某一列变量进行降权处理,从而造成平差结果不合理甚至错误,称之为虚假稳健估计现象。鉴于此,提出了多变量稳健总体最小二乘平差方法,并导出了相应的参数估计与精度评定公式。该方法对含有粗差的多变量EIV模型的各列独立变量分别进行降权处理,从而避免虚假稳健估计现象的发生。仿真算例结果表明,当观测值含有粗差时,该方法能够有效避免虚假稳健估计现象的发生,并能够定位出粗差所对应的误差方程;相较于总体最小二乘和稳健最小二乘方法,该方法的参数估计结果更接近真值。     

基于非线性高斯-赫尔默特模型的结构总体最小二乘法

变量误差（error-in-variables,EIV）模型的系数矩阵存在结构特征的情况,并且这种结构特征可以扩展到观测向量中。首先采用变量投影法将系数矩阵的增广矩阵展开成仿射矩阵形式,提取系数矩阵和观测向量中的随机量,并将EIV模型表示为非线性高斯-赫尔默特模型,然后利用非线性最小二乘原理推导了一种结构总体最小二乘法。该算法统一了普通的结构总体最小二乘法、结构数据最小二乘法以及最小二乘法。将该算法应用到真实算例和模拟算例中,两个算例结果表明,该算法与已有能够解决EIV模型结构特征的结构或加权总体最小二乘法估计结果一致,验证了该算法的有效性。同时,该算法对结构特征的提取方式简单、规律性强且易于编程实现;且在算法设计中,把结构总体最小二乘问题转换为附有参数的条件平差问题,即将其纳入到最小二乘平差理论体系,便于其扩展应用。同时对平面拟合问题的误差估计特性进行了定性分析,由分析可知参数的相对大小对估计误差的一致性有直接影响,这说明EIV模型下系数矩阵和观测向量中随机量的估计误差与真误差的一致性关系相对复杂。     

加权整体最小二乘EIV模型与算法——与"加权整体最小二乘EIO模型与算法"一文的讨论

加权整体最小二乘方法是一种能同时顾及EIV（errors-in-variables）模型中系数矩阵和观测向量误差的参数估计方法。根据不同的应用场景,EIV模型则表现出不同的结构特征。"加权整体最小二乘EIO模型与算法"一文采用EIO模型处理EIV模型中的结构化问题^*。为了将其与现有方法进行对比,本文罗列出4种处理EIV模型结构特征的方法,并归纳了8种参数估计公式。同时从精度评定的角度讨论了整体最小二乘解的一阶及更高阶精度近似评定方法。需要强调的是,针对EIV模型及其参数估计理论可以从函数模型、随机模型和参数估计方法3个方面展开研究,但各方法殊途同归。     

基于EIV模型的稳健估计

EIV（error-in-variables）模型同时考虑观测向量和系数矩阵的误差,自提出以来便得到广泛应用。目前针对EIV模型的整体最小二乘解法（TLS）假设观测值仅含有偶然误差,当观测值存在粗差时其解并不是最优的。文中通过选定合适的权函数,结合加权整体最小二乘迭代算法,导出基于EIV模型的稳健整体最小二乘迭代解法（RTLS）。线性拟合实验表明,文中方法能对粗差进行定位,且估计量受粗差影响较小,具有稳健性。     

An Aggregate Constraint Method for Inequality-constrained Least Squares Problems

The inequality-constrained least squares (ICLS) problem can be solved by the simplex algorithm of quadratic programming. The ICLS problem may also be reformulated as a Bayesian problem and solved by using the Bayesian principle. This paper proposes using the aggregate constraint method of non-linear programming to solve the ICLS problem by converting many inequality constraints into one equality constraint, which is a basic augmented Lagrangean algorithm for deriving the solution to equality-constrained non-linear programming problems. Since the new approach finds the active constraints, we can derive the approximate algorithm-dependent statistical properties of the solution. As a result, some conclusions about the superiority of the estimator can be approximately made. Two simulated examples are given to show how to compute the approximate statistical properties and to show that the reasonable inequality constraints can improve the results of geodetic network with an ill-conditioned normal matrix.     

Partial EIV模型的解法

提出了一种求解partial errors-in-variables(partial EIV)模型的思路。通过对partial EIV模型的部分元素进行移项,重组成新形式下的平差函数模型,两次运用间接平差原理分别求解平差参数与系数矩阵中的随机元素,把总体最小二乘平差问题转化为最小二乘平差问题,并通过适当变换提高了新解法的收敛速度。最后分别采用实测数据和模拟数据进行验证,求解了本文算法与已有算法的估值结果。算例结果表明,本文算法能取得与已有算法相同的结果,是切实可行的。     

不等式约束加权整体最小二乘的凝聚函数法

误差向量的方差-协方差阵是一般对称正定矩阵下的附不等式约束加权整体最小二乘平差模型,研究了其参数估计和精度评定问题。首先,将残差平方和极小化函数在整体最小二乘准则下转化为只包含模型参数的目标函数,同时将所有的不等式约束表示成一个等价的凝聚约束函数,并运用乘子罚函数策略将不等式约束加权整体最小二乘平差问题转化为相应的无约束最优化问题,并用BFGS方法求解。然后,将误差方程和约束函数线性展开,推导了最优解和观测量间的近似线性函数关系,运用方差-协方差传播律得到了最优解的近似方差。最后,用数值实例验证了方法的有效性和可行性。     

Weighted total least squares: necessary and sufficient conditions, fixed and random parameters

A standard errors-in-variables (EIV) model refers to a Gauss–Markov model with an uncertain model matrix from a geodetic perspective. Least squares within the EIV model is usually called the total least squares (TLS) technique because of its symmetrical adjustment. However, the solutions and computational advantages of the weighted TLS problem with a general weight matrix (WTLS) are mostly unknown. In this study, the WTLS problem was solved using three different approaches: iterative methods based on the normal equation, the iteratively linearized Gauss–Helmert model with algebraic Jacobian matrices, and numerical analysis. Furthermore, sufficient conditions for WTLS optimization were investigated systematically as proposed solutions yield only necessary conditions for optimality. A WTLS solution was considered to treat random parameters within the EIV model. Last, applications to test these novel algorithms are presented.     

动态EIV模型及其总体卡尔曼滤波方法

针对求解动态EIV模型时未考虑状态方程中状态转移矩阵误差的问题,本文建立了一种能够同时顾及状态方程和观测方程中各量误差的动态EIV模型。推导了针对该动态EIV模型的总体卡尔曼滤波方法及其近似精度评定公式。对比分析了本文总体卡尔曼滤波方法与已有总体卡尔曼滤波方法及总体最小二乘方法的异同。算例结果表明,本文方法统计上要优于标准卡尔曼滤波方法和已有的总体卡尔曼滤波方法。     

隐式标度因子的总体最小二乘估计方法

分析指出了标度总体最小二乘方法(STLS)存在的问题,提出了一种隐式标度因子的标度总体最小二乘方法(Im STLS)。区别于现有STLS方法在平差准则中引入标度因子,Im STLS方法在EIV函数模型中顾及标度因子,从而解决了现有STLS平差准则形式与标度因子实际表征的平差结果不一致的问题。此外,利用所建函数模型的重构表达式推导的Im STLS估计量及其方差-协方差阵,与经典最小二乘平差理论具有形式同构性。最后,验证了所提方法统一表达LS,DLS和TLS的正确性,并讨论给出了标度因子对平差结果的影响及确定方法。     

坐标转换Partial-EIV总体最小二乘方法

在测量数据处理过程中,针对系数矩阵中同时存在随机元素和固定元素的情况,Xu等通过将随机元素分离使EIV模型推广到Partial-EIV模型,并给出基于Partial-EIV模型的总体最小二乘(TLS)算法。文中介绍该算法,并将其应用在平面及空间的坐标转换中,通过与最小二乘(LS)、总体最小二乘(TLS)及加权总体最小二乘(WTLS)方法的比较和分析,验证该算法有效性。     

等式约束对病态问题的影响及约束正则化方法

有效利用参数间已知的等式约束信息能够提高最小二乘解的精度,消除秩亏,但是等式约束能否消除或减弱平差模型的病态性尚不明了,由此提出了一种通过消除部分参数将等式约束病态问题转化为无约束问题的方法。然后分析了等式约束对病态问题的影响,用简单实例证明了加入约束后,系统可能呈现良态或病态,它的性态由原设计阵和等式约束共同决定,并提出了求解等式约束病态问题的诊断-正则化两步方法。最后用一个数值实例验证了该方法的可行性。     

基于变量投影的结构总体最小二乘算法

测绘领域诸多实际应用中系数矩阵和观测向量具有结构特征,即系数矩阵和观测向量中包含固定量(甚至固定列)和随机量,并且不同位置的随机量线性相关。针对这个问题,从变量误差(errors-in-variables,EIV)函数模型出发,首先,将系数矩阵和观测向量构成的增广矩阵表示为仿射函数形式,并采用变量投影法对函数模型进行重构;然后,利用拉格朗日法推导出了一种结构总体最小二乘(structured total least squares,STLS)估计算法。算例分析结果表明,该算法与已有能够解决系数矩阵和观测向量存在结构特征的加权或结构总体最小二乘算法估计结果一致,说明了该算法的有效性,同时阐明了该算法与已有相关算法的关系。     

病态加权总体最小二乘的广义岭估计解法

针对加权情形下的变量误差(EIV)模型,采用广义岭估计法处理总体最小二乘平差的病态性问题. 结合最优化准则和协方差传播率推导了未知参数的改正数求解公式;根据参数估计值的均方误差最小化原理,通过求偏导数列出广义岭估计中岭参数的迭代解式,并讨论了广义岭参数的含义和作用,给出了确定岭参数的L-曲线法. 通过算例比较分析了加权最小二乘估计、总体最小二乘估计、加权最小二乘岭估计、总体最小二乘岭估计、加权最小二乘的广义岭估计和总体最小二乘广义岭估计,叙述了加权总体最小二乘的广义岭估计的优缺点.      

通用EIV平差模型及其加权整体最小二乘估计

以平差基本理论为基础,提出了EIV(errors-in-variables)平差模型的通用形式,涵盖了间接平差、条件平差、附有参数的条件平差及附有限制条件的间接平差等基本EIV模型形式。基于整体最小二乘估计准则,研究了通用EIV模型的加权整体最小二乘算法,并推导了估计结果的近似精度公式。通用EIV模型及其整体最小二乘算法是对EIV模型估计理论的进一步完善,统一的整体最小二乘算法有利于软件的编程实现,有助于推动EIV模型估计理论的应用。