# Resolution explained for magnetic angle sensors

Because optical encoders are functionally similar, “resolution” is often used as a main characteristic for magnetic encoders, a device providing the angle of a magnet attached to a mechanical shaft under a digital form. Resolution is a key parameter since it indicates the smallest angle that the sensor can resolve. Unfortunately, when comparing products, users may often be misled, as resolution is defined in different ways in both commercial and technical documents.

This article proposes a way to define resolution such that it can be consistently determined across different sensor’s datasheets. We will also show that for magnetic encoders, resolution alone is not sufficient to adequately compare products. Sensor bandwidth, which is often absent in many magnetic position sensor datasheets, is also required to compare magnetic angle sensors.

Measurement error

Before defining resolution, it’s important to clarify some points related to measurement error. A measurement error is defined as the difference between the measured value of a quantity and its true value. This error can be divided in two components, described below:

Systematic (or bias) error: Systematic error is the component that stays constant across multiple measurements performed under the same conditions. This error can be estimated as the difference between the average of a large number of measurements and the true value of the measurement.
Random error: Random error is the total error minus the systematic error. It accounts for the unpredictable variations in a group of measurements performed under the same conditions.

Figure 1 shows the different combinations of random and systematic error. There are three groups of measurements, with different amounts of random and systematic error. Group A has a larger random error, group B has a larger systematic error, and group C has similar random and systematic errors.

Figure 1 Combinations of random and systematic errors are segmented into three groups. Source: Monolithic Power Systems

In magnetic angle sensor datasheets, the systematic and random errors are expressed as INL and resolution, respectively. For simplicity, this article will assume that the sensor has no systematic error, which means that the average value is the true value.

Standard deviation and confidence

A metric that can be used to quantify the amount of random error in a measurement is standard deviation (σ). In statistics, σ measures the dispersion of a set of samples around their average. The higher the dispersion, the higher the σ. This parameter is also referred to as the root-mean-square (RMS) noise.

Sets of measurements often follow a bell-shaped curve distribution, also referred to as a Gaussian or normal curve (see Figure 2). This is the case when the random variation does not depend on the past error. The Gaussian curve peaks at the measurements’ average (µ), and σ characterizes its width. If the total area under the Gaussian curve is normalized to 1, then the area delimited by a range of values [a1, a2] is the probability that the result of a measurement falls somewhere between a1 and a2. The larger the range, the greater the confidence that a single measurement falls into that range.

Figure 2 Gaussian distribution is shown with µ = 0 and σ = 1. Source: Monolithic Power Systems

Table 1 lists the probability or confidence that the measurement is included into the range [µ – nσ, µ + nσ].

Table 1 The list shows the confidence factor for several values of n. Source: Monolithic Power Systems

Defining resolution

The U.S. National Institute of Standards and Technology (NIST) defines resolution as “the ability of the measurement system to detect and faithfully indicate small changes in the characteristic of the measurement result.”

Resolution is then the smallest interval that an instrument can detect. To determine this interval, this article will assume that the distribution of random errors follows the Gaussian distribution. This leads to a question: how far apart should two angles be to distinguish both angles with a reasonably high probability?

When the distance between the two angles is smaller than 6σ, the two noise distributions centered on the angles significantly overlap (denoted as “A” in Figure 3). If the result of the measurement falls in the overlap region, it’s impossible to know if the true angle is angle a1 or a2. It is only when the distance between the two angles is equal to or larger than 6σ that a single measurement can distinguish between these two points with a confidence equal to or higher than 99.73% (denoted as “B” in Figure 3). Therefore, the sensor’s resolution is an interval of 6σ.

Figure 3 Sample contained in the 6σ intervals is centered on µ1. Source: Monolithic Power Systems

Analog-to-digital conversion

Generally, the output of position sensors is given in a digital format; for example, it may be provided through an ABZ or SPI interface. In this scenario, the analog signal from the magnetic sensor must be digitized. Figure 4 shows a simplified block diagram of a digital magnetic angle sensor. Note that the diagram includes a filter block, which will be further discussed in the next section.

Figure 4 The simplified block diagram of a digital magnetic angle sensor includes a filter block. Source: Monolithic Power Systems

The step size for analog-to-digital conversion (ADC)—namely the range of values in the analog domain divided by the number of steps in the digital domain—is often wrongly interpreted as the sensor’s resolution. This interpretation is only correct when the peak-to-peak noise of the analog signal is smaller than the step size of the ADC.

However, this is not true in most scenarios. The peak-to-peak noise of the analog signal often exceeds the ADC step, so it appears at the sensor’s digital output as random flicker of the output’s least significant bits (LSBs). This is why ADC manufacturers define indicators such as “noise-free resolution” or “peak-to-peak resolution”.

Figure 5 shows how noise is carried from the analog domain to the digital domain. In this example, the step size is 1, while the peak-to-peak noise is 6. In addition, the continuous and discrete distributions are shown on the X-axis and Y-axis, respectively. Since the noise exceeds the digital step, decreasing the step size does not improve the resolution.

Figure 5 Here is how noise is carried from analog to digital domain. Source: Monolithic Power Systems

When providing a measurement in a digital format, the resolution can also be expressed in bits, calculated with Equation (1):

resolutionbit = log2 FS/6σ              (1)

Where FS is the full scale of the quantity to be measured.

In the case of angle measurements, FS = 360°, which means the resolution can be estimated with Equation (2):

resolutionbit = log2 360/6σ           (2)

Bandwidth review

When discussing sensor performance, a key parameter that is often overlooked is the bandwidth, also known as the cutoff frequency. Sensor bandwidth corresponds to a signal’s frequency range, which can be measured by the sensor. Signals with a frequency larger than the sensor bandwidth are attenuated. A detailed characterization of the sensor would require its transfer function under an analytical or graphical form. At the bare minimum, the cutoff frequency should be provided.

Figure 4 shows that a low-pass filter stage can be implemented in the sensor. This reduces the noise on the sensor output. In this case, the sensor bandwidth is same as the filter’s bandwidth. If the noise distribution is Gaussian, decreasing the filter bandwidth by a factor of 4 decreases the noise by a factor 2, which increases the resolution by 1 bit. This means that information regarding the noise or resolution should correspond with information regarding the bandwidth.

A bandwidth that is too low for an application can have dramatic effects. If the sensor is used inside a control loop, the system may be unstable, and the motor may exhibit oscillations, noise, and/or loss of efficiency (see Figure 6). In this figure, R is the position reference, AM is the motor shaft angle, and AS is the sensor output. A common design rule is to have filter bandwidth at least 10 times larger than the bandwidth of the control system or control loop.

Figure 6 Bandwidth has a significant impact on motor control loop. Source: Monolithic Power Systems

Figure 7, Figure 8, and Figure 9 show the effects of a low-pass filter bandwidth (BW) on angle measurement, noise, and control loop performance, respectively.

Figure 7 shows that the motor shaft angle and sensor output under a high BW filter almost overlap (denoted with the blue and green lines, respectively). Meanwhile, the sensor output with a lower BW filter is not able to follow the motor shaft position as accurately (denoted with the red line).

Figure 7 Different filter bandwidths can overlap and affect sensor output. Source: Monolithic Power Systems

Using a BW filter for the angle significantly reduces the noise (see Figure 8). As the bandwidth gets lower, the noise is more attenuated.

Figure 8 Different filter bandwidths can also impact sensor output noise. Source: Monolithic Power Systems

Figure 9 shows how different filter bandwidths affect motor control loop performance. If a filter has a lower bandwidth (denoted with the red line), then there is more overshoot and a longer settling time.

Figure 9 This is how different filter bandwidths can affect motor control loop performance. Source: Monolithic Power Systems

What to look for in datasheets

To ensure that a sensor is well-suited for your application, it’s crucial to make a distinction between a digital step and the actual sensor resolution.

Often, when terms such as SPI resolution, ADC resolution, and ABZ resolution are used, they specify how many bits are used for the digital representation of the measurement, but not the actual sensor resolution.

If specs such as RMS noise, peak-to-peak noise, angle noise, or noise density are included in the sensor datasheet, they are usually the most reliable source to obtain the sensor resolution. Then designers can use Equation (1) to calculate the resolution expressed in bits.

Table 2 shows an example from a datasheet. In this scenario, the resolution is misleading since it’s actually referring to the digital step. If the filter bandwidth is configurable, multiple noise values may be listed.

Table 2 Datasheet example shows how resolution figures can be misleading. Source: Monolithic Power Systems

Using the lowest noise value shown in the table, the resolution can be calculated with Equation (3):

resolutionbit = log2 360/6σ = log2 360/0.06 = 12.55            (3)

By comparing both resolution and bandwidth, it’s possible to determine the real performance differences between products. The filter bandwidth can be expressed through several parameters such as time constant, step response, or cutoff frequency. Table 2 shows an example using the filter time constant and filter cutoff frequency.

Resolution and bandwidth main considerations

For many Monolithic Power Systems (MPS) angle sensors, the digital representation of the data is 16 bits; meanwhile, the resolution, sensing technique (Hall or TMR), and the filter bandwidth vary between parts. Table 3 lists the resolution and bandwidth values for some sensors in the MagAlpha family. Note that some sensors have a configurable filter bandwidth, which allows them to be adapted to different application requirements.

Table 3 The list shows the resolution and bandwidth values for various sensors. Source: Monolithic Power Systems

The table also illustrates how the tunneling magnetoresistance (TMR) sensing technique used inside the MA600 sensor helps achieve robust resolution at higher bandwidths, when compared to Hall-based sensors.

The article explained the definition of resolution starting from the description of random error and the statistical concepts of standard deviation and confidence. It also clarified the difference between the digital representation—number of bits provided on the sensor output—and the resolution of the sensor measurement, in case it’s provided in digital form.

By showing the effect of filtering, the article proved that both resolution and bandwidth must be considered to determine a product’s real performance. Finally, it provided an example of a typical magnetic angle sensor datasheet to show how to correctly interpret the information contained in it.

Carmine Fiore is an application engineer at Monolithic Power Systems.

Serge Reymond is sensors application manager at Monolithic Power Systems.

Related Content

<!–