A GMM-based nonlinear transformation has been proposed using Radial Basis Function (RBF) networks by [6], which is a refinement of the Baudoin's model [7]. RBF network is a nonlinear interpolation technique posing the property of best approximation as shown in [8]. Figure 2 shows the structure of a typical three-layer RBF network with p inputs, one hidden layer containing m nodes corresponding to m basis functions, and q outputs. The input vector x = [1,x₁,x₂,^...,x_p] is applied to all the basis functions in the hidden layer. The output of each hidden node h_i (x), (i = 1, ^..., m) is a nonlinear function called a basis or response function. The output of the hidden layer is weighted by a weighted vector w_k = [w_k0,w_k1, ^..., w_km]^T (note that w_k0 is the bias term). Then, the output of the network y = [y₁,y₂, ^..., y_q is a weighted sum determined by

where h()x is the (m+1)-dimensional vector and W is a (m + 1) • q weight matrix.

Figure 2: 
Structure of an RBF network

In RBF networks, the choice of basis functions plays an important role in the success of approximation problems. The widely used basis functions include Gaussian functions and spline functions whose parameters are determined empirically as in [7]. In this paper, unlike Baudoin's model, we use a more principled RBF approach presented by [9] in which the basis functions are the normal probability density functions of the GMM of the source speaker. Specifically, we fix the number of basis functions as the number of mixtures of a GMM and then estimate the parameters of the GMM using the EM algorithm. The basis functions then have the form

Note that Equation (7) differs from Equation (2) in the constant term 1/ ((2π)^p/2|Σ|^1/2) since the basis functions need to be normalized.

The drawback of the RBF method is that it may be difficult to approximate a complex relationship by a global nonlinear function. Therefore, in the next section we propose a refined approach where we approximate the relationship between source and target features by a mixture of locally nonlinear functions, each of which is approximated by an RBF.

Figure 3: 
Transformation function using Piecewise RBF. Nonlinear functions are adopted locally. w_i is a linear transformation of h(_x) and (ν_i,Γ_i)

In this section, w e propose a more generalized version of the LSE method where each local linear function is substituted by a local nonlinear function which is modeled by an RBF network. However, since it is hard to determine a different set of basis functions for each local RBF, we use the same set of basis functions h(x) = [1,h₁(x,h₂(x,^...,h_m(x) for every local RBF (see Figure 2). Similar to the RBF transformation method above, here the basis function h_m(x, (i = 1,^...,m) is the “normalized” Gaussian distribution density in Equation (7). Consequently, the transformation function has the following form

The transformation function F( • ) is entirely defined by the p-dimensional vectors ν_i and the P × p matrices Γ_i for i = 1,^...,m.

These parameters are estimated by linear least squares estimation on the training data so as to minimize the total error

Specially, the least squares optimization of the parameters is the solution of the following set of linear equations

for all t = 1,^...,n. In the matrix form, (11) can be written as

Where the two matrix ν and Γ are the unknown parameters of the transformation function. ν is a m × p matrix, Γ is a m² × p matrix and Y is a n × p matrix containing the target spectral vectors as follows

P is a n × m posterior matrix and Δ is a n × m² matrix that depends on the conditional probabilities, the source vectors, and the GMM parameters.

The solution for Equation (11) is given by the normal equation

The left most matrix in Equation (13) is symmetric but not positive definite and thus cannot be inverted using the Cholesky decomposition. Therefore, we exploit SVD to compute its pseudo inverse.

We use the MOCHA-TIMIT speech database [8] to train and evaluate proposed system. For training, 30 sentences were used for each speaker which result in more than 6000 vectors. 10 sentences were used for evaluation. Two male and two female speakers are participated in the experiments. We perform eight conversion tasks, four male-to-female, and four female-to-male conversions. Table 1 show the speaker combination used in our experiments. There are two important distances in voice conversion: the transformation error E(t(n), ) and the inter-speaker error E(s(n), t(n) where s(n), t(n, denote the source, target, converted speech respectively. All the errors are conceptual and cannot be measured directly. In this experiment, these errors are approximated by objective measures as

where A,B is the two sequences of LSF vectors, n is the number of vectors in each vector sequence, p is the order of LPC and L^i,k is the kth component of ith LSF vector.

To take into account the inter-speaker errors, we define the LSF performance index as Equation (15).

The Performance index defined as Equation (15) uses normalized error to compare the performance of different voice conversion tasks across the different speaker combinations. The performance index P_LSF is 0 for a simple copy of source speech to the output without conversion. In the case of producing the exact target speech, P_LSF is 1. Although E_LSF and P_LSF are not the standard measures of error between two speech signals, they can be applied to input and output parameters of the conversion system directly. This is the reason why we use them as objective measures.

In the experiments, we investigated the influence of the number of mixture components m on the performance of conversion system. LSE and RBF were used as baseline systems and compared with proposed piecewise RBF method. Experiments were performed while increasing m as 1, 2, 5, 8, 16, 32, 64, 128 with fixed LPC order p = 16.

Table 2 shows the experimental results averaged over all speaker combinations. Experimental results show that when the number of mixtures is increased, proposed method gives better results than the baseline systems. This can be interpreted as for the small number of mixtures, linear transformation methods gives better results, however for the large number of mixtures, local non-linear transformation function gives better conversion results by removing the drawbacks of linear and global non-linear transform functions.