ambiguity jumps in AquaDopp HR
In a deployment with Nortek AquaDopp HR, we used a maximum horizontal velocity value of 0.49 m/s(given in the software) which seems like it is lower than some of the values HR faced. Because data have some irregularities in profile and checking with the Beam Data shows that there are ambiguity jumps, similar problem discussed in the thread below:
http://www.nortek-as.com/cgi-bin....=6;t=22
We are a little bit confused with the discussions made in the above thread.
1) Firstly, what is exactly the case called as 'large' jump , other than spikes between Vmax and Vmin values? Because, there are spikes which do not reach to the peak value in Beam coordinates, but they are also certainly spikes and need to be corrected probably.
2) How can we calculate this ambiguity velocity for a specific case? Checking with the literature and information related to some similar instruments do not give consistent values. It seems like that it has a formula depending on max.velocity settings and sound speed (and in my opinion distance to bottom). Bcause in the thread above, an ambiguity velocity of 20.3 cm/s is suggested for a speed of sound of 1500 m/s in the case of a 30 cm/s configuration. If it does not have a certain formula, what should we take it for maximum velocity of 0.49 m/s. ?
3)After a jump(either positive or negative ), if the velocity does not turn back to its original sign at the next time step, what should we do to this value? Shift in a way that it will be aligned with the previous value, or anything else? The attached file(a part of beam1 velocity time series of a cell of HR in the related deployment) shows a time series which is hard to predict whether the jumps are in positive or negative directions and to postprocessing it seems like too hard.
4) In order to avoid these jumps and therefore these difficulties in the post-processing, is it enough to deploy HR to regions that we are sure that there will not be velocities greater than its maximum value or are there any other considerations?
Anybody have a suggestion or met with the same problems before?
Thanks,
Ilgar
There is a paper on our WEB site that you may find of use:
HR-NDP paper from Switzerland
It describes the predecessor to the Aquadopp but it has a section about how the system works that may be of interest.
As for the ambiguity velocity, it is a function of spacing between the two transmit pulses ("lag"). This lag has to be larger than the profiling range so in this sense the profiling range is the key parameter that determines the amb. vel. In other words, you can avoid the problem by reducing the profiling range enough that you are sure that the local velocity is smaller than the ambiguity velocity.
The Vector/Vectrino is different in the sense that the profiling range does not enter into the picture. In these instruments, the velocity ranges are there simply to allow the user to minimize the random noise in his velocity data.
Best regards, Atle Lohrmann
PS I realize that it is not simple to understand how these systems work. The best solution is thus to adjust the profiling range and avoid situations where the actual velocity exceeds the ambiguity velocity
" />
'>
Thank you very much for your urgent answer and helps upto now. I still fail to comprehend some points from your answer and references I went through. From our previous talks and the thread below:
http://www.nortek-as.com/cgi-bin....=6;t=22
there is a certain Vmax or amb.vel. value for each case that you can process and filter your data with;
if the jump is negative, then
newvel = oldvel + 2*ambiguity velocity
else
newvel = oldvel - 2*ambiguity velocity
end
So the first issue is how to calculate this ambiguity velocity used in the loop above.
Lastly, in a time series similar to the one in the attached file, is there a way-easy or difficult, no matters- to determine which of these are the jumps and which are the valid data? (I know that it may be hard to comment from a small portion of data but just as an example) Because, it makes jumps in negative and positive direction. Or should we say that such a data set, containing timeseries with too much of such behaviour, is useless?
Suggestions of users with similar problem will also be appreciated.
Regards,
Ilgar
Hi,
I'm having a similar problem to that discussed above. I'm using an Aquadopp profiler HR with a custom head. We are sampling only a single vertical beam.
The expected vertical velocity range is 0.27 m/s. Nonetheless, this value is routinely exceeded in the data. Sometimes apparently unaffected by this range limitation, and sometimes wrapped.
1) I don't understand why, since we are only sampling one beam in beam coordinates, the velocities shouldn't always be between the minimum velocity and maximum velocity -0.27<W<0.27. That is, when a velocity w exceeds 0.27 shouldn't it be: w_measured = w_real - 2*0.27.
I have attached an example timeseries to illustrate my point. Aquadopp_output.jpg shows velocities in excess of the maximum expected value at ~t=38s. Later at t=~54s, a similar peak is wrapped. I have many more examples available.
2) What determines when velocities are wrapped, and when they are not?
3) What exactly does checking the "Extended velocity range" box when doing the deployment planning do? Is this related to the problem I'm having?
Thanks,
Peter
Hi Peter,
I'll attempt to answer your questions but someone from Nortek-AS will be able to provide a bit more information about how exactly the extended velocity range processing is handled. I think my response below is accurate, but I will let them confirm.
1) I don't understand why, since we are only sampling one beam in beam coordinates, the velocities shouldn't always be between the minimum velocity and maximum velocity -0.27<W<0.27. That is, when a velocity w exceeds 0.27 shouldn't it be: w_measured = w_real - 2*0.27.
As you've noted, the biggest problem with pulse coherent processing is ambiguity in the determined velocity which is related to the cross-correlation method used to determine the Doppler shift. In the most basic pulse coherent processing there are two pings with a known lag between them. The velocity will wrap around as described by Atle earlier in the forum when it exceeds the ambiguity velocity for a given lag. This happens because Fourier transforms are used to calculate the cross-correlation, and they treat all signals as periodic. If the change in phase between the two pulses is large enough or exceeds the Nyquist criteria, the change can be interpreted as either a really large forward shift or a negative shift.
Hi Peter,
The scheme we have developed for extending the velocity range consist of a second pulse with a shorter lag since we need a shorter lag to get to a higher velocity range. But since the lag is shorter we can do longer profile over the whole profiling range in a pulse coherent way. Instead we place a single velocity cell towards the end of this shorter lag, the measured data from this velocity cell is what is output in the .hr2 ASCII file.
Now, how do we extend the velocity of the whole profile from that? Since the velocity cell from the second lag gives us a point 1/3 into the profile, with three times the velocity range, we start at the corresponding velocity cell in the actual profile and resolve the ambiguities using our extended velocity from the second lag. We then resolve the ambiguities for all cells in the profile moving from this point towards the transducer and towards the end of the profile. The assumption here is that the variation between neighbouring cells are less than half an ambiguity. So as long as that assumption is true, you can see why you can get velocities that are outside the expected velocity range (which is a bonus anyway).
Your example time series shows good data most of the time, the wrap you see around t=~54s probably comes from a corresponding wrap in the measured cell from the shorter lag. You can consult the hr2-file to verify this. Maybe you can post the corresponding time series from the hr2-file here? In addition there seems to be a few times where the variation between neighbouring cells is too large for the ambiguity resolution scheme. This is probably due to drops in the correlation. So either you need to increase the correlation my matching the cell size and sampling rate to the turbulence level at the site, or you could correct this using time series analysis.
If the velocity range for the shorter lag is exceeded, that phase wrap will of course cause a corresponding wrap in the whole profile. You should then decrease the profiling range in order to increase the velocity range.
Best regards,
Sven Nylund
Hi,
1) There are many dead links here (e.g http://www.nortek-as.com/cgi-bin....=6;t=22 , http://www.nortek-as.com/knowledge-center/forum/current-profilers-and-current-meters/841376305/view) and elsewhere. Can someone at Nortek please fix these.
2) Ambiguity: I have attached a plot (hr2_aa.pdf) of data from an .hr2 file for a single burst (512 data points) out from a long dataset. The extended velocity range option was used and, per the .hdr file (had_2M01.hdr), the maximum velocities are 0.54 and 0.23 m/s for the horizontal and vertical components, respectively. From the plot of traces of the X, Y, Z velocities(hr2_aa.pdf center panel) it appears that wrapping for all components (X, Y, Z) happens at 0.23 m/s. Thus, is the ambiguity to be taken as the smaller of the vertical or horizontal range in the .hdr file, i.e. 0.23 (vertical range) for all velocity components?
3) Attached is the respective plot of the X, Y, Z velocity components (vel_aa.pdf) for all the profiles of the same burst shown above. It seems that even when only the vertical component shows wrapping (e.g. see hr2_aa.pdf at 16:24:14-16:24:14 and 16:24:42-16:24:44), it affects all velocity components (including horizontal) in all cells (see profiles ~110-120 and ~340-360, respectively, in vel_aa.pdf). Similarly, wrapping in a horizontal component (.e.g. X component. hr2_aa.pdf plot at 16:24:08-16:24:10) seems to affect both the X and the vertical components of the respective profiles (profiles ~65-80 in vel_aa.pdf).
4) How, if at all, can data such as shown in the above be fixed and still be used?
5) There are ("regular"?) spikes apparent in the traces of profile velocities in "non-wrapped" regions (e.g. profiles ~10-65 in vel_aa.pdf, which refers to the period ~16:24:01-16:24:08 in hr2_aa.pdf; see particularly Vx). It seems that the signal in this region is good (>100) and the correlation is >70. Thus, what is the possible reason for these spikes? Any suggestions how these may be fixed / taken care of?
NOTE: since the max. allowed attachment is 100 KB, the attachments for the above will be in two separate compressed files. See next message.
Best,
Ayal
Hi Ayal,
I'm not sure if there is a way to update the links automatically so they might remain broken. A while ago the Forums were moved to a new system and the broken links are unfortunately a result of that.
As for your remaining questions, I'll address them below.
Sven and I have been looking at ambiguity wrapping and discovered a small error in Firmware Version 1.08 where the HR2 data is actually reported in beam coordinates regardless of the coordinate system for profile data. Your header file indicates you were using Firmware Version 1.06 and based on the plot of data you submitted I suspect the *.hr2 data is reported in your chosen coordinate system (XYZ). Otherwise, I would expect each of the three velocities from the *.hr2 file to look more similar than they do.
It is important to know your coordinate system when discussing ambiguity wrapping as you want to make sure you are working in beam coordinates when checking for wrapping and fixing it, it really just makes life much simpler.
The two reported velocity ranges (horizontal and vertical) are actually estimates based on the beam ambiguity velocity and the beam angle. I need to check with Sven on exactly how these are reported in the header file, but based on intuition, you can see the vertical velocity range is going to be close to the beam velocity range since the beam is more closely aligned to vertical. So, both of the estimates are valid, but it is important to understand what the instrument thinks is horizontal and vertical since it's tied to the beam angle relative to the instrument case and not to the earth. I realize that's confusing and I am working on getting a better description of this for you and other users out soon.
Because wrapping occurs in a single beam and each beam generally figures into the transformed orthogonal velocities, wrapping in a single beam can affect the transformed orthogonal velocities. Examine the transformation matrix in the header file and you'll see that with the exception of the second row, all three beams are needed to estimate the X and Z velocities. The Y velocity can be determined without the information from Beam 1 and would be unaffected by wrapping there (the first element of row 2 is zero indicating Beam 1 is not involved in calculating the Y velocity). Long story short, the wrapping you are seeing in your profile data is a result of probably one beam being affected, which based on your velocity time series is probably the one most closely aligned with the direction of wave propogation. Your *.hr2 data shows wrapping in all three beams because it is transformed to XYZ coordinates.
To fix wrapping, I recommend working in beam coordinates, so you will need to transform you XYZ data (including the *.hr2 data) to beam coordinates, which is actually rather easy once you work through the trigonometry and algebra once. Here is the NortekAS Library entry containing a MATLAB script on how to do this: http://www.nortek-as.com/en/library/forum-attachments/coordinate-transformation/view. I also recommend the following reference: http://ams.allenpress.com/perlserv/?request=get-abstract&issn=1520-0426&volume=007&issue=01&page=0019 where the full transformation scheme is discussed in the Appendix.
Once in beam coordinates, it is merely a matter of identifying the regions where wrapping is occurring and correcting them as discussed above. This again isn't too difficult, but can take a few minutes to wrap your head around to make sure you are doing it correctly (at least it takes a little time for me).
The spikes you are referring to are potentially caused by wrapping (just not nearly as blatant as the obvious ones). There are a lot of other reasons they can show up and are a fact of life when working with Doppler systems. There are many schemes to eliminate them and I discuss a general procedure here: http://www.nortek-as.com/en/knowledge-center/forum/hr-profilers/441224288. As always, make sure what you are throwing away is really bad data and you're not massaging the data too much to make it look like you want it to 
I hope the above helps and I will hopefully have a followup post to clarify some of the above points early next week.
P.J.
Hello Peter,
Thanks for your detailed reply. In fact , the whole discussion above brings me to another question that has been bugging me for a while now - wouldn't it be better to always (?; and even for non HR modes) to use beam coordinates and then use e.g. a Matlab script to convert to XYZ, or ENU, if so desired? I am not sure how/when this coordinate transformation is done in Nortek's software, i.e. at what stage is the transformation from beam to ENU applied. Is it done when converting the data using Data Conversion option in Aquapro? At any other stage?
Are there any Matlab, or other, scripts that you know of that will help to "fix" wrapping problems? (This, perhaps, sounds wishful, but maybe a good soul at Nortek or elsewhere has written one.)
BTW, when using Firefox, both link http://www.nortek-as.com/en/library/forum-attachments/coordinate-transformation/view and link http://www.nortek-as.com/en/knowledge-center/forum/hr-profilers/441224288 result in:
We're sorry, but that page doesn't exist…
and then:
You might have been looking for...
However, the links are found OK when I used Opera (as you know Opera is made by a Norwegian company...)
P.S. in which versions is there a problem with the reported coordinate system and is this only for the HR2 data file? Since v. 1.08 is on the website - has it been fixed in the v 1.08 that is posted?
Best,
Ayal
Hi Ayal,
Wether you collect in beam coordinates or not depends entirely on your application needs. Mean flows should be robustly measured by a well set up instrument regardless of the coordinate system, but to measure turbulence it is better not to average since this will reduce the variance, and since the transform to XYZ or ENU coordinates is essentially a spatial average, a data set in XYZ coordinates will only pick up larger scale turbulent motions. I would recommend working in beam coordinates only if you need to or expect problems (for instance measurements at a new unknown site) or are interested in turbulence.
On instrument, the coordinate transform is done towards the end of processing. I'm not sure if it's done before or after any instrument ensemble averaging, although I have a feeling it is nearly the last step in the on board processing. It's not done by the AquaPro HR software to my knowledge.
I'm not aware of any MATLAB scripts to perform unwrapping of acoustic Doppler data, but I also tend to write most of my own analysis code so I haven't looked very hard. Ullrich Lemmin has a nice paper on a scheme developed for his ADVP (http://www.iop.org/EJ/article/0957-0233/17/2/012/mst6_2_012.pdf?request-id=2888e62d-4a64-4124-bfb6-a69d76920d6b) which might give you some more insight into this problem.
The current firmware version on the NortekAS website is 1.08. The data I have exported from an HR Profiler running this firmware used beam coordinates for the *.HR2 file. This should be the default for all future versions of the *.HR2 data according to Sven, but note that the Data File Description section of the header file implies it will be in the user selected coordinate system.
Last, the two links you provided work for me in Safari 4, so it looks like this is a problem with Firefox. I don't know enough about web browsers to say why this would be or definitely that this is the case.
P.J.
Hi Peter,
I guessed pretty much the same as you did above regarding timing of coordinate transformation in the processing stream, but I hope that someone from NORTEK-AS will provide some further info on the issue.
Thanks for the ref. to the paper. In fact I have downloaded it earlier today and it is on my "soon to read" list.
Regarding coordinate system/s for *.HR2 and the profile files, i.e. *.v1, *.v2, *.v3 files - I hope that the AquaDopp HR manual will be updated to reflect this, even for the sole purpose of avoiding further nagging by me...
Best,
Ayal
Hi Ayal,
The coordinate transformation in Nortek instruments is done at the same interval as specified by the compass update rate. For the HR this is currently once every average/sample interval. For 1 Hz - 8 Hz sample rate in burst mode, where the compass update rate is grayed out, the compass is read at the same rate. Our instruments transform the data to the coordinate system specified in the configuration (deployment planning) for the regular velocity data. For the hr2 data in the HR we will make sure that the software reflects the actual coordinate system in the data file format section of the hdr-file.
Like Peter says the coordinate transformation in the HR is one of the last steps in the processing. As long as the coordinate transformation is done at the same rate as the velocity output you can always change coordinate system in post processing without any loss of information.
Best regards,
Sven
As long as the instruments is not moving between compass updates...
The original argument against collecing data in beam coordinates came from systems mounted on mooring lines (i.e. sensor is moving). If you spin around 360 degrees during the averaging interval, the current would appear to be zero if you averaged in beam (or XYZ) coordinates
Best regards, Atle
Sven,
"For the hr2 data in the HR we will make sure that the software reflects the actual coordinate system in the data file format section of the hdr-file."
Does this suggest that the .hr2 data is (as of some revised version of v 1.08) now recording data in say, ENU, if I had selected that in the deployment setup? How can I differentiate between v1.08 that defaulted to recording the .hr2 file in BEAM coordinates versus the 'new' v1.08? This is the output listed in the .hdr for .hr2:
7 Velocity (Beam1|X|East) (m/s)
8 Velocity (Beam2|Y|North) (m/s)
9 Velocity (Beam3|Z|Up) (m/s)
Thanks,
-Chris
Previously Sven Nylund wrote:
Hi Ayal,
The coordinate transformation in Nortek instruments is done at the same interval as specified by the compass update rate. For the HR this is currently once every average/sample interval. For 1 Hz - 8 Hz sample rate in burst mode, where the compass update rate is grayed out, the compass is read at the same rate. Our instruments transform the data to the coordinate system specified in the configuration (deployment planning) for the regular velocity data. For the hr2 data in the HR we will make sure that the software reflects the actual coordinate system in the data file format section of the hdr-file.
Like Peter says the coordinate transformation in the HR is one of the last steps in the processing. As long as the coordinate transformation is done at the same rate as the velocity output you can always change coordinate system in post processing without any loss of information.
Best regards,
Sven
Hi Chris,
The software and firmware versions used by the data file and instrument are stored in the binary data file and will be reported in the header file after data conversion. Software version should be around line 38 and the firmware version will be in the "Hardware Configuration" section around line 85.
I don't have access to the release notes for the firmware, but this is where I believe this coordinate system for the EVR data will be set (I also didn't see a mention of this change in the software release notes for version 1.06). So, for now I would assume the coordinate system is beam if your firmware version is 1.08. Sven should be along to clarify this in a bit.
In the meantime, you can do a simple check of your data by plotting the EVR data from the *.hr2 file. If it's in beam coordinates all three velocities will be roughly the same order magnitude and appear fairly similar in structure. If it's XYZ or ENU, the third (vertical) component will generally be much smaller than the other two, i.e. it will look at lot more like what you would expect from an orthogonal coordinate system in most flows.
P.J.
Forgot your password?
New user?
