Tips, tricks and notes

List of posts:
  • How to organize your digital working environment: two principles and one example for beginners This post is for those who start an internship or a PhD. It summarizes a couple of advices I give to my students. 2 principles: To start with, here are ...
    Posted Jul 8, 2013, 12:28 PM by Guillaume Maze
  • How to create a clean netcdf file using the built-in toolbox of Matlab A while ago I wrote a post on how to create a clean netcdf file from Matlab using the Unidata netcdf toolbox. Starting from release 2010a, Matlab is now delivered ...
    Posted Feb 14, 2013, 3:37 AM by Guillaume Maze
  • Vi editor help I don't use the vi/vim editor much and I found myself constantly looking for useful commands. Here they are, thanks to lagmonster.org.
    Posted Feb 13, 2013, 5:28 AM by Guillaume Maze
  • jQuery UI widget template/snippet for TextMate In a recent project I came to write a couple of jQuery UI widgets to build the user interface of a web API.  I thus set up a clean jQuery ...
    Posted Jun 6, 2012, 6:39 AM by Guillaume Maze
  • How to download and map Argo profiles for an ocean using Matlab ? This tutorial will describe how to download and plot Argo profiles (and a density map) for a given ocean. This tutorial is for people who have a very small experience ...
    Posted Jun 6, 2012, 5:47 AM by G. Maze ‎(admin)‎
  • Multiple pages PDF document with Preview version 5.0.3 I've been using a Mac and Preview to visualize PDF documents. I loved taking advantage of the sidebar to create/cut/mix pages in documents. As easy as a ...
    Posted Jun 6, 2012, 6:36 AM by Guillaume Maze
  • Google Chrome extension and web apps I've been a Firefox user for several years, and to be honest, since I had my first Sun station to work with ! I moved to Mac OS and kept ...
    Posted Feb 13, 2013, 5:31 AM by Guillaume Maze
  • About Matlab figure export in pdf/eps I export my Matlab figures in the pdf format and this produces a nice a4 or usletter size pdf document with my plot. In fact, Matlab creates a pdf of ...
    Posted Feb 13, 2013, 5:33 AM by Guillaume Maze
  • GFD Search Engine (add to Firefox custom search) I started aggregating a bunch of useful sites to look for GFD informations on the web, some kind of specialized Google search engine.You can find its homepage here:http ...
    Posted Apr 6, 2010, 1:52 AM by G. Maze ‎(admin)‎
  • Online access to articles with your library and proxy (bookmarklet to modify url) MIT provides a very nice method to access articles from publishers. It's through their library proxy. I used to use a simple trick, simply adding .libproxy.mit.edu at ...
    Posted Sep 23, 2010, 3:11 AM by G. Maze ‎(admin)‎
  • How to create a clean netcdf file from Matlab ? Update: For those using Matlab version 2010b and higher a netcdf set of routines is now incorporated in the distribution. The syntax is different and the following script is then ...
    Posted Apr 23, 2016, 9:23 AM by Guillaume Maze
  • Intempestive ssh deconnection For a reason I don't know my internet provider kills ssh connections on port 22 after a short inactive delay, like a couple a minutes.This is very annoying ...
    Posted Apr 6, 2010, 12:50 AM by G. Maze ‎(admin)‎
  • Simple creation of movies I had some difficulties to create movies from Matlab watchable from every OS platform, even from Mac Leopard to Tiger.I finally adopted a simple approach, not based on the ...
    Posted Sep 22, 2010, 2:09 AM by G. Maze ‎(admin)‎
  • Read PDF directly into your Firefox window with Mac Simply install the Add-on located here:http://code.google.com/p/firefox-mac-pdf/
    Posted Apr 30, 2009, 12:41 AM by G. Maze ‎(admin)‎
  • Apache Access Control I use MAMP to emulate an Apache web server on my laptop. Although my local webpages are for intranet local use only, producing a first level access control, I may ...
    Posted Apr 28, 2009, 6:25 AM by G. Maze ‎(admin)‎
  • SSH No more passwords !Create a public/private key set with the command:ssh-keygen -t rsathen add the public key under ~/.ssh/id_rsa.pub to the remote file ...
    Posted Apr 27, 2009, 7:22 AM by G. Maze ‎(admin)‎
  • Crontab Edit the crontab with:crontab -ethen in the file add something according to the rules:# +---------------- minute (0 - 59)# |  +------------- hour (0 - 23)# |  |  +---------- day of month (1 - 31)# |  |  |  +------- month (1 - 12 ...
    Posted Apr 27, 2009, 7:10 AM by G. Maze ‎(admin)‎
  • Editing Firefox personalized dictionnary It happens quite often to add to the dictionary a word incorrectly identified as misspelled. But it also happens to go to fast and add a wrong word.To edit ...
    Posted Apr 27, 2009, 7:05 AM by G. Maze ‎(admin)‎
Showing posts 1 - 18 of 18. View more »


How to organize your digital working environment: two principles and one example for beginners

posted Jul 8, 2013, 12:28 PM by Guillaume Maze

This post is for those who start an internship or a PhD. It summarizes a couple of advices I give to my students.

2 principles:

To start with, here are two principles that should guide you:
  1. "In 4 years, someone I don't know now should be able to easily find information in my stuff". This means that you should consider that you work as part of a team. In practice, it means for example: extensively comment you script, add readme.txt files to your folders to explain what's inside or where it comes from, don't use words like "toto","tutu","tata","tmp","old","very_old","new" for files or folders for more than a week, name files and folders for what they are or contain, etc ...
  2. "Think to future tasks in order to organize present ones". I think the core principle here is modularity. It applies to folder's organization as well as code's structure. For instance, if you analyze a given dataset for a given project, the scripts for the analysis should be in a project folder and the data files in another. May be later, you'll conduct the same analysis on a different dataset or you use the same dataset to conduct a different analysis. In a code structure, this means that you should use functions and/or classes as soon as you copy/pasted the same code snippet more than 3 times !

1 example:

To set-up a clean working environment, you first need a root folder. It could be your home directory (it's often the case), but it could also be any folder from your disk. Just select it and place EVERYTHING underneath. Side effects will be a much easier handling of backups and a clean split between your work and private life if you use a single computer. Now that you have your root folder, let's take an example to illustrate the two principles stated above. Let's say you start an internship in a lab. You're going to use Matlab to conduct your analysis and visualization and you're going to use a dataset called Argo downloaded from a ftp server. Under the root folder, I suggest you have the following folders:
+ data
+ matlab
+ projects
Now, let's expand these folders:
- data
  - ARGO
    readme.txt
    + ftp
    + docs
    - matlab
      plot_one_float_trajectory.m
- matlab
    m_map -> m_map_1.4
  + m_map_1.2
  + m_map_1.4
  + copoda
  startup.m
- projects
  - my_first_analysis_of_Argo_data
    - analysis
      + figures
      load_coordinates.m
      map_profiles.m
    - reports
      + figures
      version0.doc
  + my_second_project_as_a_phd
Under the data folder, we have a folder dedicated to the Argo dataset. It contains:
  • a readme.txt where you would say on which ftp server you downloaded the dataset and when
  • a ftp folder where the data files are
  • a docs folder where you will put documents related to this dataset such as the file format description for instance
  • a matlab folder where you will place scripts that are not related to a scientific analysis or a specific project, but are related to generic tasks on the dataset
Whatever your use or analysis of this dataset, this folder is a standalone source for it. You can then share this folder to your team or get back to it in 4 years easily. Matlab has its own folder at the root level because in this way you will be able to handle your own toolboxes and customized environment easily. In this example I added several sub-folders.The "m_map" folder is in fact a link to the "m_map_1.4" one. This simple trick could be use to simply handle several versions and updates to a similar toolbox. Everywhere you need to refer to this toolbox, you point to the "m_map" folder (see startup.m file below). Then, with the link you point to the desired version. The "copoda" folder doesn't make use of this method because it's a toolbox handled using a svn repository. Versioning is thus delocalized. Last, you have the "startup.m" file. It will automatically be run by Matlab when starting a new session. In this file you will tell Matlab to load any toolbox. It could look like:
% Startup file for Matlab
% Last update: 2013/07/01
disp('Hello you !')

% Load toolboxes:
addpath('~/matlab/m_map')

% End of startup
Last, the Projects folder will contain all the real science work ! If there should be only thing to back up here, it would be this folder; all the others could be regenerated. Note that I created two sub-folders for the first project: "analysis" and "reports". The folder "analysis" will contain all the scripts and figures produced during the research phase, the folder "reports" will contain all the reports and hopefully articles about your work. Of course, all of this is an example to help beginners organize their work a minima. It all depends on your tools and common practices in your lab, although the two principles stated above should be kept in mind whatever your situation.

How to create a clean netcdf file using the built-in toolbox of Matlab

posted Feb 13, 2013, 6:05 AM by Guillaume Maze   [ updated Feb 14, 2013, 3:37 AM ]

A while ago I wrote a post on how to create a clean netcdf file from Matlab using the Unidata netcdf toolbox. Starting from release 2010a, Matlab is now delivered with a built-in netcdf toolbox. And the API is not quite similar. So, I thought it would be nice top update the how-to as well. Here it is…





This is a Matlab script showing how to create a clean netcdf (4) file, including groups.

Create dummy data to put into the netcdf file

% Define axis, for example:
X = 0:2:360; % Longitude
Y = -90:2:90; % Latitude
Z = -5000:100:0; % Depth
T = datenum(1958:2006,1,1,0,0,0); % Time

% Create a random field to record in the cdf file dimensions are (Z,Y,X):
C = rand(length(Z),length(Y),length(X));

File standard set-up

Open file and insert dimensions

Note I exclude the temporal axis for now. So let’s create a new netCDF4 file:

scope = netcdf.create('myfirstcdffile.nc','netcdf4');

I call the netcdf pointer a scope and not ncid or nc because in netCDF4, we can create groups with their own dimensions and variables. So we need to know where we are when defining dimensions and so on. It will become clear in the next section.

% Define useful constants:
NC_GLOBAL = netcdf.getConstant('NC_GLOBAL');
fillValue = -9999;

% Define dimensions:
dimidZ = netcdf.defDim(scope,'depth',length(Z));
dimidY = netcdf.defDim(scope,'latitude',length(Y));
dimidX = netcdf.defDim(scope,'longitude',length(X));

% Define axis:
varid = netcdf.defVar(scope,'depth','double',[dimidZ]);         
netcdf.putAtt(scope,varid,'standard_name','depth');
netcdf.putAtt(scope,varid,'long_name','Vertical distance below the surface');
netcdf.putAtt(scope,varid,'units','m');
netcdf.defVarFill(scope,varid,false,fillValue);
netcdf.putVar(scope,varid,Z);

varid = netcdf.defVar(scope,'latitude','double',[dimidY]);
netcdf.putAtt(scope,varid,'standard_name','latitude');
netcdf.putAtt(scope,varid,'long_name','Station latitude');
netcdf.putAtt(scope,varid,'units','degrees_north');
netcdf.defVarFill(scope,varid,false,fillValue);
netcdf.putVar(scope,varid,Y);

varid = netcdf.defVar(scope,'longitude','double',[dimidX]);
netcdf.putAtt(scope,varid,'standard_name','longitude');
netcdf.putAtt(scope,varid,'long_name','Station longitude');
netcdf.putAtt(scope,varid,'units','degrees_east');
netcdf.defVarFill(scope,varid,false,fillValue);
netcdf.putVar(scope,varid,X);

Insert global attributes

These are just my stuff, you need to customize. But note attributes related to the convention. This is very useful for you netcdf file to be read by anybody else than you.

netcdf.putAtt(scope,NC_GLOBAL,'title','My first netcdf file')
netcdf.putAtt(scope,NC_GLOBAL,'long_title','Self explanatory, isnt''it ?')
netcdf.putAtt(scope,NC_GLOBAL,'comments','All variables are just noise !')
netcdf.putAtt(scope,NC_GLOBAL,'institution','LPO/Ifremer')
netcdf.putAtt(scope,NC_GLOBAL,'source','my_model')

netcdf.putAtt(scope,NC_GLOBAL,'Conventions','CF-1.6')   
netcdf.putAtt(scope,NC_GLOBAL,'Conventions_help','http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.6/cf-conventions.html')

netcdf.putAtt(scope,NC_GLOBAL,'CreationDate',datestr(now,'yyyy/mm/dd HH:MM:SS'))
netcdf.putAtt(scope,NC_GLOBAL,'CreatedBy',getenv('LOGNAME'))
%netcdf.putAtt(scope,NC_GLOBAL,'MatlabSource',which('your_matlab_script_generating_this_file'))

Check what we’ve just done

We can now close the netcdf file (optional):

netcdf.close(scope)

and go in a terminal to look at it with:

ncdump -h myfirstcdffile.nc

or simply from Matlab display a minimum of informations:

>> ncdisp('myfirstcdffile.nc')
Source:
           /Users/gmaze/work/Production/How to : Tutorials/myfirstcdffile.nc
Format:
           netcdf4
Variables:
    depth    
           Size:       51x1
           Dimensions: depth
           Datatype:   double
    latitude 
           Size:       91x1
           Dimensions: latitude
           Datatype:   double
    longitude
           Size:       181x1
           Dimensions: longitude
           Datatype:   double

or a maximum:

>> ncdisp('myfirstcdffile.nc','/','full')
Source:
           /Users/gmaze/work/Production/How to : Tutorials/myfirstcdffile.nc
Format:
           netcdf4
Global Attributes:
           title            = 'My first netcdf file'
           long_title       = 'Self explanatory, isnt'it ?'
           comments         = 'All variables are just noise !'
           institution      = 'LPO/Ifremer'
           source           = 'my_model'
           Conventions      = 'CF-1.6'
           Conventions_help = 'http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.6/cf-conventions.html'
           CreationDate     = '2013/02/14 11:03:09'
           CreatedBy        = 'gmaze'
Dimensions:
           depth     = 51
           latitude  = 91
           longitude = 181
Variables:
    depth    
           Size:       51x1
           Dimensions: depth
           Datatype:   double
           Attributes:
                       standard_name = 'depth'
                       long_name     = 'Vertical distance below the surface'
                       units         = 'm'
                       _FillValue    = -1e+04
    latitude 
           Size:       91x1
           Dimensions: latitude
           Datatype:   double
           Attributes:
                       standard_name = 'latitude'
                       long_name     = 'Station latitude'
                       units         = 'degrees_north'
                       _FillValue    = -1e+04
    longitude
           Size:       181x1
           Dimensions: longitude
           Datatype:   double
           Attributes:
                       standard_name = 'longitude'
                       long_name     = 'Station longitude'
                       units         = 'degrees_east'
                       _FillValue    = -1e+04

Note that the ncdisp function can be used only with the file name. Here I used the complete syntax to highlight the specificity of netCDF4, with the slash /, we ask for information at the top location of the file. We don’t have groups yet, but we could have also asked for informations about a specific group.

Insert a 3D field

If the file was closed, we need to reopen it:

scope = netcdf.open('myfirstcdffile.nc','WRITE')  % Rq: here we use 'WRITE' because the file already exists

The variable we want to record is in the Matlab field C but we have to give it a name and define metadata for the netcdf file:

varname = 'my_first_field';
long_name = 'A long description of what this is, well simply noise !';
unit = 'm/s'; % for example of course

And now we can write it:

varid = netcdf.defVar(scope,varname,'double',[dimidZ, dimidY, dimidX]);
netcdf.putAtt(scope,varid,'name',varname);
netcdf.putAtt(scope,varid,'long_name',long_name);
netcdf.putAtt(scope,varid,'units',unit);
netcdf.defVarFill(scope,varid,false,fillValue);
%netcdf.defVarDeflate(scope,varid,true,true,9);

v = C;
    v(isnan(v)) = fillValue;
    netcdf.putVar(scope,varid,v);

We can now close the netcdf file (optional):

netcdf.close(scope)

and from Matlab:

ncdisp('myfirstcdffile.nc')

and see the new variable we just added !

Insert a 2D field

The procedure is similar to the 3D case, except that when defining the new variable the dimensions are different:

iz = 1; % Define the level
varname = sprintf('%s_level%i',varname,iz);
long_name = 'Still a long description of what this is, well simply noise !';
unit = 'm/s';
varid = netcdf.defVar(scope,varname,'double',[dimidY, dimidX]);
netcdf.putAtt(scope,varid,'name',varname);
netcdf.putAtt(scope,varid,'long_name',long_name);
netcdf.putAtt(scope,varid,'units',unit);
netcdf.defVarFill(scope,varid,false,fillValue);
v = squeeze(C(iz,:,:));
    v(isnan(v)) = fillValue;
    netcdf.putVar(scope,varid,v);

Handling the time axis

You probably use the native Matlab format for your date/time information: datenum. But this is not how conventional netcdf handles time.

Insert the new dimension

Usually, I define my netcdf time axis as the days (can be fractional) since 1900–01–01. So let’s reopen the file:

scope = netcdf.open('myfirstcdffile.nc','WRITE');  % Rq: here we use 'WRITE' because the file already exists

and add the new time axis. First, define the dimension:

dimidT = netcdf.defDim(scope,'time',length(T));

Then add the variable:

varid = netcdf.defVar(scope,'time','double',[dimidT]);          
netcdf.putAtt(scope,varid,'standard_name','time');
netcdf.putAtt(scope,varid,'long_name','Time of measurements');
netcdf.putAtt(scope,varid,'units','days since 1900-01-01 00:00:00');
netcdf.defVarFill(scope,varid,false,fillValue);
netcdf.putVar(scope,varid,T-datenum(1900,1,1,0,0,0));

NOTE THAT THIS IS ONLY MY CHOICE, YOU NEED TO CUSTOMIZE WITH YOUR REQUIREMENTS

Insert a time varying field

Now we create a dummy variable, supposedly a timeserie of an horizontal 2D fields:

D = rand(length(T),length(Y),length(X));

And record it:

varname = 'my_time_serie';
long_name = 'This is a time serie of noise !';
unit = 'degC';

varid = netcdf.defVar(scope,varname,'double',[dimidT,dimdY,dimdX]);         
netcdf.putAtt(scope,varid,'long_name',long_name);
netcdf.putAtt(scope,varid,'units',unit);
netcdf.defVarFill(scope,varid,false,fillValue);
v = D;
    v(isnan(v)) = fillValue;
    netcdf.putVar(scope,varid,v);   

We can now close the netcdf file (optional):

netcdf.close(scope)

and from Matlab:

ncdisp('myfirstcdffile.nc')

Variant with groups

This is one of the main new possibility of netCDF4: you can put some structure into the netcdf file itself. I won’t go into the details, but this simply like sub-folders into the file. Let’s see with an example. Imagine that we want we the same netcdf file to contain model output (or observations) as well as some results of their analysis, like say a principal component timeserie from an EOF analysis. We could define the netCDF4 file structure like:

global_scope  = netcdf.create('myfirstcdf4file.nc','netcdf4');
netcdf.putAtt(global_scope,NC_GLOBAL,'Comment','This file is not a mess ! It is organized by group')

data_scope    = netcdf.defGrp(global_scope,'Raw_data');
netcdf.putAtt(data_scope,NC_GLOBAL,'Comment','This group or part of the file contains raw data')

another_scope = netcdf.defGrp(global_scope,'Analysis');
netcdf.putAtt(another_scope,NC_GLOBAL,'Comment','This second group or part of the file contains analysis data')

and try:

ncdisp('myfirstcdf4file.nc')

wait ! this throw an error:

Error using netcdflib
The NetCDF library encountered an error during execution of 'open' function - 'Unknown file format (NC_ENOTNC)'.

Error in netcdf.open (line 60)
        [varargout{:}] = netcdflib ( 'open', filename, varargin{1} );

Error in internal.matlab.imagesci.nc/openToRead (line 1258)
            this.ncRootid = netcdf.open(this.Filename,'NOWRITE');

Error in internal.matlab.imagesci.nc (line 122)
                    this.openToRead();

Error in ncdisp (line 51)
ncObj   = internal.matlab.imagesci.nc(ncFile);

Ok, this was simply to show you that there is a difference between what Matlab knows or wants to do with the actual file state on disk. We need to synchronize both with a sync:

netcdf.sync(global_scope)
ncdisp('myfirstcdf4file.nc')

which returns:

Source:
           /Users/gmaze/work/Production/How to : Tutorials/myfirstcdf4file.nc
Format:
           netcdf4
Groups:
    /Raw_data/

    /Analysis/

this is where the ncdisp parameter location is useful, we can display information for a single group, try, for instance:

ncdisp('myfirstcdf4file.nc','Raw_data','full')

Now, each group can contain its own dimensions and variables, and filling in the data, is as simple as with the previous case, we just need to specify the correct scope of where we want to put information. Consider the following script implementing our use case scenario:

%% Setup data for entire file:

netcdf.putAtt(global_scope,NC_GLOBAL,'title','My first netCDF4 file')
netcdf.putAtt(global_scope,NC_GLOBAL,'long_title','Self explanatory, isnt''it ?')
netcdf.putAtt(global_scope,NC_GLOBAL,'comments','Groups are very useful !')
netcdf.putAtt(global_scope,NC_GLOBAL,'institution','LPO/Ifremer')
netcdf.putAtt(global_scope,NC_GLOBAL,'source','my_model')
netcdf.putAtt(global_scope,NC_GLOBAL,'Conventions','CF-1.6')    
netcdf.putAtt(global_scope,NC_GLOBAL,'Conventions_help','http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.4/cf-conventions.html')
netcdf.putAtt(global_scope,NC_GLOBAL,'CreationDate',datestr(now,'yyyy/mm/dd HH:MM:SS'))
netcdf.putAtt(global_scope,NC_GLOBAL,'CreatedBy',getenv('LOGNAME'))
%netcdf.putAtt(global_scope,NC_GLOBAL,'MatlabSource',which('your_matlab_script_generating_this_file'))

%% Put data into the first group:

% Define dimensions:
dimidT = netcdf.defDim(data_scope,'time',length(T));    
dimidY = netcdf.defDim(data_scope,'latitude',length(Y));
dimidX = netcdf.defDim(data_scope,'longitude',length(X));

% Define axis:
varid = netcdf.defVar(data_scope,'time','double',[dimidT]);         
netcdf.putAtt(data_scope,varid,'standard_name','time');
netcdf.putAtt(data_scope,varid,'long_name','Time of measurements');
netcdf.putAtt(data_scope,varid,'units','days since 1900-01-01 00:00:00');
netcdf.defVarFill(data_scope,varid,false,fillValue);
netcdf.putVar(data_scope,varid,T-datenum(1900,1,1,0,0,0));

varid = netcdf.defVar(data_scope,'latitude','double',[dimidY]);
netcdf.putAtt(data_scope,varid,'standard_name','latitude');
netcdf.putAtt(data_scope,varid,'long_name','Station latitude');
netcdf.putAtt(data_scope,varid,'units','degrees_north');
netcdf.defVarFill(data_scope,varid,false,fillValue);
netcdf.putVar(data_scope,varid,Y);

varid = netcdf.defVar(data_scope,'longitude','double',[dimidX]);
netcdf.putAtt(data_scope,varid,'standard_name','longitude');
netcdf.putAtt(data_scope,varid,'long_name','Station longitude');
netcdf.putAtt(data_scope,varid,'units','degrees_east');
netcdf.defVarFill(data_scope,varid,false,fillValue);
netcdf.putVar(data_scope,varid,X);

% Insert data:
varid = netcdf.defVar(data_scope,'SST','double',[dimidT, dimidY, dimidX]);
netcdf.putAtt(data_scope,varid,'name','Sea_Surface_Temperature');
netcdf.putAtt(data_scope,varid,'long_name','Temperature at the surface of the ocean');
netcdf.putAtt(data_scope,varid,'units','degC');
netcdf.defVarFill(data_scope,varid,false,fillValue);
v = D;
    v(isnan(v)) = fillValue;
    netcdf.putVar(data_scope,varid,v);


% Synchronize:
netcdf.sync(data_scope)

% Look at what we've just done:
disp('------ ncdisp of file (after data insertion in the global scope and one group):')
ncdisp('myfirstcdf4file.nc','/','full')
disp('------')

Now, let’s put data into the second group. Image we performed an EOF analysis only over the first 40 snapshots of the variable named SST and recorded in the Raw_data group. We want to save the first principal component of the analysis:

%% Put data into the second group:

% Define dimensions:
T2 = T(1:40);
dimidT = netcdf.defDim(another_scope,'time',length(T2));    

% Define axis:
varid = netcdf.defVar(another_scope,'time','double',[dimidT]);          
netcdf.putAtt(another_scope,varid,'standard_name','time');
netcdf.putAtt(another_scope,varid,'units','days since 1900-01-01 00:00:00');
netcdf.defVarFill(another_scope,varid,false,fillValue);
netcdf.putVar(another_scope,varid,T2-datenum(1900,1,1,0,0,0));

% Insert data:
varid = netcdf.defVar(another_scope,'PC','double',[dimidT]);
netcdf.putAtt(another_scope,varid,'name','Principal_Component');
netcdf.putAtt(another_scope,varid,'units','none');
netcdf.defVarFill(another_scope,varid,false,fillValue);
v = rand(length(T2),1);
    v(isnan(v)) = fillValue;
    netcdf.putVar(another_scope,varid,v);

% Synchronize:
netcdf.sync(another_scope)

Note how a dimension and a variable can be defined in 2 groups and have different properties. Before netCDF4 this would have forced us to create a dimension and a variable with a different name. So the final structure of our netCDF4 file is:

>> ncdisp('myfirstcdf4file.nc','/','full')

Source:
           /Users/gmaze/work/Production/How to : Tutorials/myfirstcdf4file.nc
Format:
           netcdf4
Global Attributes:
           Comment          = 'This file is not a mess ! It is organized by group'
           title            = 'My first netCDF4 file'
           long_title       = 'Self explanatory, isnt'it ?'
           comments         = 'Groups are very useful !'
           institution      = 'LPO/Ifremer'
           source           = 'my_model'
           Conventions      = 'CF-1.6'
           Conventions_help = 'http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.4/cf-conventions.html'
           CreationDate     = '2013/02/14 12:21:35'
           CreatedBy        = 'gmaze'
Groups:
    /Raw_data/
        Attributes:
                   Comment = 'This group or part of the file contains raw data'
        Dimensions:
                   time      = 49
                   latitude  = 91
                   longitude = 181
        Variables:
            time     
                   Size:       49x1
                   Dimensions: time
                   Datatype:   double
                   Attributes:
                               standard_name = 'time'
                               long_name     = 'Time of measurements'
                               units         = 'days since 1900-01-01 00:00:00'
                               _FillValue    = -1e+04
            latitude 
                   Size:       91x1
                   Dimensions: latitude
                   Datatype:   double
                   Attributes:
                               standard_name = 'latitude'
                               long_name     = 'Station latitude'
                               units         = 'degrees_north'
                               _FillValue    = -1e+04
            longitude
                   Size:       181x1
                   Dimensions: longitude
                   Datatype:   double
                   Attributes:
                               standard_name = 'longitude'
                               long_name     = 'Station longitude'
                               units         = 'degrees_east'
                               _FillValue    = -1e+04
            SST      
                   Size:       49x91x181
                   Dimensions: time,latitude,longitude
                   Datatype:   double
                   Attributes:
                               name       = 'Sea_Surface_Temperature'
                               long_name  = 'Temperature at the surface of the ocean'
                               units      = 'degC'
                               _FillValue = -1e+04

    /Analysis/
        Attributes:
                   Comment = 'This second group or part of the file contains analysis data'
        Dimensions:
                   time = 40
        Variables:
            time
                   Size:       40x1
                   Dimensions: time
                   Datatype:   double
                   Attributes:
                               standard_name = 'time'
                               long_name     = 'Time of measurements'
                               units         = 'days since 1900-01-01 00:00:00'
                               _FillValue    = -1e+04
            PC  
                   Size:       40x1
                   Dimensions: time
                   Datatype:   double
                   Attributes:
                               name       = 'Principal_Component'
                               units      = ''
                               _FillValue = -1e+04

Vi editor help

posted Dec 6, 2012, 1:59 AM by Guillaume Maze   [ updated Feb 13, 2013, 5:28 AM ]

vim logo
I don't use the vi/vim editor much and I found myself constantly looking for useful commands. Here they are, thanks to lagmonster.org.


jQuery UI widget template/snippet for TextMate

posted Jun 6, 2012, 4:15 AM by Guillaume Maze   [ updated Jun 6, 2012, 6:39 AM ]

In a recent project I came to write a couple of jQuery UI widgets to build the user interface of a web API. 
I thus set up a clean jQuery UI widget snippet for my favorite text editor TextMate to easily create widgets. 
Note that you can easily modify the source code to create a template instead a of snippet.
To install the new snippet, simply doucle click on the following tmSnippet file:



Note:
The scope selector is source.js.jquery provided by the excellent jQuery TextMate bundle (by K. Swedberg and J. Chaffer), but you could prefer source.js ...

Otherwise, here is the code (for some reasons tabulations do not appear bellow, but the snippet file is fully tabulated):
/*

jQuery UI Widget Plugin
Namespace: ${1:namespace}
Widget:    ${2:widgetname}
Version:   ${3:version}
Create a widget instance via: 
\$( "#something" ).$2()
The plugin instance is accessible via: 
\$( "#something" ).data("$2")

Widget public methods accessible via:
\$( "#something" ).$2("SomePublicMethod")
\$( "#something" ).data("$2").SomePublicMethod()

Set-up Widget options via:
\$( "#something" ).$2("option","OptionName",OptionValue)
\$( "#something" ).$2("option",{OptionName: OptionValue, OptionName: OptionValue})

Standard Methods:
\$( "#something" ).$2("destroy")
\$( "#something" ).$2("widget")
Creation date: `date +%Y-%m-%d`
Author: ${TM_SHORT_NAME} (${TM_AFFILL})
Contact: ${TM_COORD}
*/

(function(  ) {
\$.widget( "$1.$2", {
 
/* ------------------------------------
WIDGET DEFAULTS OPTIONS
------------------------------------ */
options: { 
OptionName: 'OptionValue', // Standard option with a value
// OptionNameCallBack: {function(){}} // Option with a callback function
},
 
/* ------------------------------------
WIDGET SET UP METHOD
------------------------------------ */
_create: function() {
var self = this,
options = self.options,
titleId = \$.$1.$2.getTitleId(self.element);
// Simple set-up for the widget container:
self.uiContainer = \$('<div></div>')
.appendTo(document.body) // Append the Widget to the Document
// .appendTo(self.element) // Or Append it to the caller element
.hide()
.attr({
role: '$1-$2',
'aria-labelledby': titleId
});
return self;
},
  /* ------------------------------------
WIDGET PRIVATE METHODS
Insert your private methods with an underscore
------------------------------------ */
_PrivateMethod: function(){
var self = this,
options = self.options;
// Code here
return self;
},
 
/* ------------------------------------
WIDGET PUBLIC METHODS
------------------------------------ */
// Insert your public methods here:
SomePublicMethod: function(){
var self = this,
options = self.options;
// Code here
return self;
},
/* ------------------------------------
OPTIONS HANDLERS (PRIVATE METHODS)
------------------------------------ */
// Use the _setOption method to respond to changes to options
_setOption: function( key, value ) {
switch( key ) {
case "OptionName":
// Handle changes to OptionName
break;
}
 
// In jQuery UI 1.8, you have to manually invoke the _setOption method from the base widget
\$.Widget.prototype._setOption.apply( this, arguments );
// In jQuery UI 1.9 and above, you use the _super method instead
// this._super( "_setOption", key, value );
},
_setOptions: function( options ) {
var self = this;
\$.each( options, function( key, value ) {
self._setOptiosn( key, value );
});
},
/* ------------------------------------
WIDGET REQUIRED PUBLIC METHODS
------------------------------------ */
// Use the destroy method to clean up any modifications your widget has made to the DOM
destroy: function() {
// In jQuery UI 1.8, you must invoke the destroy method from the base widget
\$.Widget.prototype.destroy.call( this );
// In jQuery UI 1.9 and above, you would define _destroy instead of destroy and not call the base method
},
// Return the widget object:
widget: function() {
// Method to gain access to the widget created container:
return this.uiContainer;
}
});
/* ------------------------------------
WIDGET VERSION
------------------------------------ */
\$.extend(\$.$1.$2, {
version: "$3",
uuid: 0,
getTitleId: function(\$el) {
var id = \$el.attr('id');
if (!id) {
this.uuid += 1;
id = this.uuid;
}
return 'data-$2-id-' + id;
},
});
}( jQuery ) );

How to download and map Argo profiles for an ocean using Matlab ?

posted Jun 1, 2012, 6:04 AM by G. Maze ‎(admin)‎   [ updated Jun 6, 2012, 5:47 AM ]


This tutorial will describe how to download and plot Argo profiles (and a density map) for a given ocean.
This tutorial is for people who have a very small experience with Matlab.
This tutorial assumes that you want to plot Argo profiles for one of the given oceanic basin available on GDAC ftp servers under the geo folder: Atlantic, Pacific and Indian oceans.

Step 0: Setting-up your working environnement

Ok, open a terminal window, move to your favorite working folder and create a new folder, let's say 'argo_work':
mkdir argo_work
cd argo_work

Step 1: Download the Argo netcdf files (c-shell script)

We're going to use the wget program and for later reuse we're going to put all this step into a single c-shell script. The script is as follow:
#!/bin/csh
# Download all Argo netcdf profiles for a given ocean

# Define here which GDAC to use:
set gdac = 'ftp.ifremer.fr/ifremer/argo' # Use the French GDAC ftp server
#set gdac = 'usgodae.org/pub/outgoing/argo' # You can also use the US GDAC ftp server

# Define here which ocean to load:
set basin = 'indian_ocean'

# Define here where to put the data:
set folder = 'data'

# Download files:
wget -c -x -P./$folder -r -N ftp://$gdac'/geo/'$basin

# Done 
exit 0

The script starts by defining some parameters to indicate where to get the data and for which ocean, here the Indian Ocean. Note that this script will load ALL the data !
Copy/paste the code into a text editor, save the script as 'download_this' in the 'argo_work' folder, make it executable and launch it:
chmod ugo+x download_this
./download_this
All the data will go under the sub-folder named 'data'. 
The folder structure will depend on the GDAC you used, for the Ifremer one you'll get something like:
    data/ftp.ifremer.fr/ifremer/argo/geo/indian_ocean/<YEAR>/<MONTH>/<YEAR><MONTH><DAY>_prof.nc
and for the US one:
    data/usgodae.org/pub/outgoing/argo/geo/indian_ocean/<YEAR>/<MONTH>/<YEAR><MONTH><DAY>_prof.nc
where <YEAR>, <MONTH> and <DAY> are the list of available years and months while the end file is an Argo multiprofile netcdf with all profiles for a given day

Step 2: Read Argo profiles coordinates (matlab function)

Ok, so now that we have the raw material under the 'data' folder, we're going to move to Matlab scripting to do all the rest of the work.
Let's create a matlab function which is going to read all profiles coordinates (latitude,longitude,time).
To keep a clean folder architecture, in the 'argo_work' folder, creates a sub-folder where we'll put all Matlab scripts:
mkdir matlab
cd matlab
Now let's write the Matlab function. 
To do this, we create a file named like the function we want to set-up. Let's then create a file named 'read_data.m' and open it in your favorite text editor.

In the file, we need to tell Matlab that we're going to create a function. The way to do this is to embed all the code into:
function varargout = read_data(varargin)
    % All the function code goes here !
end% end function

We first need to declare variables to set a couple of informations:
%- DECLARE LOCAL VARIABLES

% Define where is the data folder relative to the Matlab script:
data_folder = fullfile('..','data'); 

% Define which ftp server we used to download the data.
% Un-comment by removing the % sign the correct line:
ftp_server = 'ifremer'; 
%ftp_server = 'us';

% Define which oceanic basin we downloaded the data for:
basin = 'indian_ocean'; % In this case the Indian ocean

% And now adjust the folder names down to basin folders:
switch ftp_server 
    case 'ifremer', data_folder = fullfile(data_folder,'ftp.ifremer.fr','ifremer','argo','geo',basin);
    case 'us',      data_folder = fullfile(data_folder,'usgodae.org','pub','outgoing','argo','geo',basin);
end% switch

Ok, with this piece of code, we know where are the YEAR/MONTH folders. Note that we used the Matlab function fullfile to create a correct path string whatever the platform we're working on. This allows to get rid of / or \ issues ...

Now we're facing multiple choices in the design of the code to scan the years/months folders. We can either:
  • look for any folder (whatever their names) or 
  • declare a list of years and months to look for in the data folder. 
I prefer the second approach for two reasons: 
  1. it's an implicit method to ensure that we're going to look and analyse data coming from the ftp server (its structure is preserved) and
  2. it provides an easy handling method if at some points we only want to load data for a given month or year.
But this choice is up to you. It is always a compromise between flexibility and security design. 

To implement the second choice we then need to declare a list of years and months we want to list the netcdf for:
% Define the list of years to scan:
year_list = [2010:2012];
% Define the list of months to scan:
month_list = [1:12];

Now that we have declared where to start the scan and some kind of iteration process, we can start to loop:
%- FILE SCANNING LOOP:

for iy = 1 : length(year_list)
    for im = 1 : length(month_list)
        % Define folder to be scanned:
        scan_folder = fullfile(data_folder,sprintf('%0.4d',year_list(iy)),sprintf('%0.2d',month_list(im)));
    end%for im
end%for iy

At this point, these two loops (over years and months) don't do anything, we'll add codes step by step.

In the scan_folder declaration we used the Matlab function sprintf in order to control the way years and months are written as string in the folder path. This is important for month's name, because for month numbers lower than 10 (January to September), the string must begins with a 0 like: '02' for February and not only '2'.

For each scan_folder, we can now use the Matlab function dir to list files present in the folder. If we find a file corresponding to an Argo Netcdf multiprofile, then we'll read the coordinates.
We scan the content of the folder with the Matlab function dir and store the result in the variable raw_file_list. This variable is a Matlab structure. A structure is not like double or cell arrays, it can contains a lot of different variable types which can be retrieved or set using an object oriented syntax. In our case, the structure returned by the function dir is a structure array (which means it's a list of elements with similar content pattern) with some useful properties like: name, dates, bytes, isdir, ... (see Matlab help for more details). We are going to use the name property to check if it's a Netcdf file we can expect in this folder:

%- FILE SCANNING LOOP:
for iy = 1 : length(year_list)
    for im = 1 : length(month_list)
            % Define folder to be scanned:
            scan_folder = fullfile(data_folder,sprintf('%0.4d',year_list(iy)),sprintf('%0.2d',month_list(im)));                        
        disp(scan_folder); % Print on screen the folder we're scanning.
            
            % List correct files in the folder:
       raw_file_list = dir(scan_folder);         
       for ifile = 1 : length(raw_file_list)

                % Is this file looking like an Argo Netcdf multiprofile ?
                read_this = false;

                % Perform tests:
if length(raw_file_list(ifile).name) == 16
if raw_file_list(ifile).name(end-2:end) == '.nc'
if raw_file_list(ifile).name(1:4) == sprintf('%0.4d',year_list(iy))
if raw_file_list(ifile).name(5:6) == sprintf('%0.2d',month_list(im))
read_this = true;
end%if
end%if
end%if
end%if

                % Read profiles coordinates out of this file:
                switch read_this
                    case false
                        % Nothing to do then
                    case true
                        % Let's read coordinates:
                end%swtich
        
       end%for ifile
        
   end%for im
end%for iy

Once the list of files in scan_folder is retrieved, we loop through all files in order to perform a couple of tests to verify the Netcdf file can be expected. 
We initiated a variable called read_this to false. We'll set it to true for a correct file. We could implement the file name test list in many different form. Here we used a very simple serie of embedded 'if' statements, we test the following:
  1. file name is 16 characters long (it must be ! otherwise it couldn't be of the form: <YEAR><MONTH><DAY>_prof.nc
  2. file extension correspond to the Argo Netcdf extension '.nc'
  3. the first 4 characters correspond to the folder year
  4. the 5th and 6th characters correspond the the folder month.
If these 4 tests are passed successfully, the variable read_this  is set to true because it looks good. 

Inside the ifile loop, if the read_this variable is set to true, we then need to open the Netcdf file and get profiles coordinates out of it. The piece of code is as follow:
% Let's read coordinates:
% Open the Netcdf file:
ncid = netcdf.open(fullfile(scan_folder,raw_file_list(ifile).name),'NC_NOWRITE');

% Read profiles latitude:
varid    = netcdf.inqVarID(ncid,'LATITUDE');
LATITUDE = netcdf.getVar(ncid,varid,'double');

% Read profiles longitude:
varid    = netcdf.inqVarID(ncid,'LONGITUDE');
LONGITUDE = netcdf.getVar(ncid,varid,'double');

% Read profiles time:
% Get the time reference:
varid = netcdf.inqVarID(ncid,'REFERENCE_DATE_TIME'); 
REFERENCE_DATE_TIME = netcdf.getVar(ncid,varid)'; 
ref = datenum(str2num(REFERENCE_DATE_TIME(1:4)),...
str2num(REFERENCE_DATE_TIME(5:6)),...
str2num(REFERENCE_DATE_TIME(7:8)),...
str2num(REFERENCE_DATE_TIME(9:10)),...
str2num(REFERENCE_DATE_TIME(11:12)),...
str2num(REFERENCE_DATE_TIME(13:14)));
% then the relative time axis:
varid = netcdf.inqVarID(ncid,'JULD'); 
JULD  = netcdf.getVar(ncid,varid);
% and finally the absolute time axis:
TIME  = ref + JULD;

% Close the Netcdf file:
netcdf.close(ncid);

% Add those profiles coordinates to the function output variables:
X = [X; LONGITUDE];
Y = [Y; LATITUDE];
T = [T; TIME];

This syntax implies that you're using a recent Matlab version, one with a built-in Netcdf api. First we open the file which is then referred to as the ncid variable pointer. Reading latitude and longitude is fairly easy as you can see. For time it's a little bit more complicated because we have to recompute the asbolute time axis from a reference and relative time axis. I won't go into details here, check out the Argo documentation for that.
In the end, we close the netcdf file (you must do this, otherwise Matlab we'll keep 'connection' open to files and at some point it will blow up your session if too many files are open) and finally concat the new profiles coordinates (LATITUDE,LONGITUDE,TIME) to variables used to output the results out of the function (X,Y,T). Those variables needs to be initiated before the time looping like:
% Initiate coordinates arrays:
X = [];
Y = [];
T = [];
%- FILE SCANNING LOOP:
    ...
We're only left with function output set up, just before the end function tag, we can insert:
% Output
varargout(1) = {X};
varargout(2) = {Y};
varargout(3) = {T};
This says that if one is requesting for outputs, we'll serve X then Y then T.
And that's it, save the file 'read_data' and you're all set. 

Oh wait ! What if we want to load profiles for a given year, or month ? We don't want to edit this function every time, we prefer to use parameters when calling it !
How can we do that ? Well, my favorite method is to insert a block of code after the 'DECLARE LOCAL VARIABLES' section of the function which is going to read variables from the function arguments and possibly overwrite some values. This can simply be achieved with:
%- LOAD USER PARAMETERS:
if nargin > 0
if mod(nargin,2) ~= 0
error('Please provide options as: ''OPTIONS NAME'',VALUE pair(s)');
end% if 
for in = 1 : 2 : nargin
eval(sprintf('%s = varargin{in+1};',varargin{in}));
end% for in
clear in
end% if
This syntax usage is simple. For instance, by default we're loading all months of any year because the variable month_list is set to 1:12. With the previous peace of code, we can call the function from the Matlab prompt (or another script) like:
>> [X,Y,T] = read_data('month_list',[12 1 2]);
and the variable month_list will be overwritten so that in this case only Dec/Jan/Feb months are loaded.
The final read_data.m file can be downloaded at the end the tutorial.

Step 3: Plot locations and compute a density map (matlab script)

Let's move on to the fun part: plotting ! Now we're going to write all the code in a Matlab file which is not a function. This means that all variables declared in the script are going to be available in the Matlab workspace. 
Requirements:
  • We are going to use the m_map Matlab toolbox to plot map easily (download the m_map package here). So first you need to install the toolbox (see their webpage).
  • The profile density map will be computed using the very nice function bin2mat created by A. Stevens and distributed on the Matlab Central file exchange platform. So next you need to put the bin2mat.m function file in your matlab folder.
First create a file named 'plot_data.m' and open it in your favorite text editor. Let's load the data (profiles coordinates) for the year 2010, then enter the line:
[X,Y,T] = read_data('year_list',2010);
You may want to start the file by a 'clear' statement so that the workspace is clear of any variables. You really don't want to enter a 'clear all' to also delete global variables.
If you now look at your workspace you have the following variables available:
>> whos 
  Name          Size             Bytes  Class     Attributes

  T         27385x1             219080  double              
  X         27385x1             219080  double              
  Y         27385x1             219080  double  
At the time of writing this tutorial (I just downloaded profiles for the Indian Ocean), we can see that 27,385 profiles were sampled in 2010 for this ocean. And this is the smallest ocean ! so be careful if you try to load profiles for multiple years or a bigger ocean, numbers can get very large and your computer may don't like it.

Number of profiles per week

First we're going to plot a very simple diagram showing the number of profiles per week (this time frame is appropriate because we loaded a single year of data).
This simplest way to do this is to use the Matlab function hist and ask the function to compute the number of profile for 52 bins (hence 52 weeks in a year). 
[nw tw] = hist(T,52);
figure;
plot(tw,nw,'.-');
grid on, box on,datetick('x')
xlabel('Time');ylabel('# of profiles');title('Number of Argo profiles per week in the Indian Ocean for 2010');
We then open a new figure, plot the number of profiles per week (nw) using the time axis returned by hist (tw), format the grid, the time axis and put some labels and a title. You should have something like:

There are about 520 profiles sampled every week in the Indian Ocean, at least in 2010. Very impressive network !

Map of profiles locations

We're going to plot a map with all the profiles locations now. We first need to initiate a projection using the m_map function m_proj:
m_proj('equid','lon',[min(X) max(X)]+[-1 1],'lat',[min(Y) max(Y)]+[-1 1]/2);
This line set up an equidistant projection system centered over a rectangular area specified by the options 'lon' and 'lat'. Note that here we're using the range of coordinates of profiles to fit the map correctly. If you're curious you can now open a new figure, type in m_grid;m_coast; at the prompt, and here we go: a map of the Indian Ocean ! The m_map toolbox is smart enough so that to plot a coastline and the grid you don't have to worry about anything as long as you're not customizing the look.
Note that the m_proj function defines global variables and needs to be called only once, you don't have to call it every time you plot a map.

How do we plot profiles locations ? Well, simply using the m_plot function:
figure;
hp = m_plot(X,Y,'+');
m_coast;m_grid;
title('Argo profiles for 2010');
And that's it.
Wait, wait, this standard figure is really ugly, we want to customize it so that profiles markers are not so big, etc ... Let's try again:
figure
hp = m_plot(X,Y,'+');
set(hp,'markersize',2,'color',[1 .5 0]);
m_coast('patch',[1 1 1]/3);
m_grid('box','fancy','xtick',[-180:30:180],'ytick',[-90:10:90]);
title('Argo profiles locations for 2010','fontweight','bold');
These are the figures I get from the two previous plots:

See how you can customize the map by looking at all the options for m_coast and m_grid. The profiles marker design is up to you as well ...

Map of profiles density

Ok, plotting profiles locations with a marker is very simple but as soon as the number of profiles increases maps become totally useless. One method out of this visualization issue is to compute and plot what we call a profile density map: ie the number of profiles for a given area, usually a rectangular cell of a couple of degrees wide.
To do such a thing we need to define a grid like for instance:
% Define a 1x1 degree grid:
dxi = 1; dyi = 1; % Grid size in longitude and latitude
xi = min(X)-1:dxi:max(X)+1; % Define grid point longitude
yi = min(Y)-.5:dyi:max(Y)+0.5; % Define grid point latitude
[Xi,Yi] = meshgrid(xi,yi); % Create the meshed grid.
And now here comes the magic ! Using the bin2mat function computing a profile density map is as simple as:
dens = bin2mat(X,Y,ones(size(X)),Xi,Yi,@sum);
The 1st/2nd arguments of bin2mat are the longitude and latitude axis of profiles, the 3rd argument is the quantity we want to bin (here we take 1 because we simply want to compute the number of points), 4th/5th arguments are the grid points coordinates and the 6th argument is the function handle you want to use. Here this command literrally means sum 1 each time you get a value in X/Y falling into a grid cell defined by Xi/Yi. 
And now we can plot the density map using m_pcolor:
figure
m_pcolor(xi,yi,dens);
m_coast('patch',[1 1 1]/3);
m_grid('box','fancy','xtick',[-180:30:180],'ytick',[-90:10:90]);
title('Argo profiles density map for 2010','fontweight','bold');
But this figure isn't showing much because one area (the Arabian Sea) has a very large number of profiles compared to the rest of the basin. So we may want to tune the color axis to highlight those poor regions as well. We can do it using the caxis Matlab function which set the range of the color axis, meaning if we use:
caxis([1 10]);
then all grid points with values higher or equal to 10 will have the same color (the last one from the colormap) while all grid points with values smaller or equal to 1 will also have the same color (the first one from the colormap). In this case, I think it would be a good practice to set-up a colorbar where the highest value color (more than 10) is significantly different than the one just smaller to clearly indicate that the color axis is saturated and that some informations are not visible in the map. If we use the standard 'jet' colormap, the last color is red. So we can change it to black and keep a nice look:
cmap = jet(64);
cmap(end,:) = [0 0 0]; % Replace the last color (red) by black
colormap(cmap);
Below are the two figures before and after the color axis and color map adjustment:
Profiles density (before) Profiles density (after)
Using a density grid of 1x1 degree may not the best choice here. Let's try the map with a 3x3 degree grid. The only thing to change in the previous code is the dxi/dyi line:
dxi = 3; dyi = 3; % Grid size in longitude and latitude
and the color axis because we're going to have 9 times the number of initial profiles per grid cell. So now the density map looks like: 

Conclusion

That's it for today. We've learn how to download profiles from a GDAC ftp server, to read those data from Matlab and plot some stuff about them. Many improvements could be added, like a test on the profiles coordinates quality or a more flexible method to list files to get the data from.
I hope you found this tutorial helpful. Feel free to post a comment if you have questions or issues...


Multiple pages PDF document with Preview version 5.0.3

posted Feb 23, 2011, 5:27 AM by Guillaume Maze   [ updated Jun 6, 2012, 6:36 AM ]


I've been using a Mac and Preview to visualize PDF documents. I loved taking advantage of the sidebar to create/cut/mix pages in documents. As easy as a simple drag/drop of any page.
Unfortunately when I update to Mac OS 10.6.6 a couple of weeks ago, I noticed a change of behavior of Preview. In previous versions you could drag and drop a new page to a pdf document and save as something new the augmented document. Doing this with the new Preview version simply add the new page to the sidebar and when saving as, only the selected page (or initial document) was saved.
Well, today I finally found how the new Preview works. Instead of dragging and dropping the new page below or between pre-existing pages (when little horizontal blue line shows up) you have to drop the file on the pdf thumbnail and only then it will add a new page to the document and save it when using 'save as'...
You'll notice the difference in the window's title. It will change 
from: "toto.pdf (1 page) (2 documents, 2 total pages)" 
to: "toto.pdf (page 1 of 2)"

Google Chrome extension and web apps

posted Sep 22, 2010, 2:10 AM by G. Maze ‎(admin)‎   [ updated Feb 13, 2013, 5:31 AM by Guillaume Maze ]

I've been a Firefox user for several years, and to be honest, since I had my first Sun station to work with ! I moved to Mac OS and kept using Firefox because I never fell in love with Safari. But as new updates keep pilling up new functions and a bunch of other stuff, I started to find Firefox buggy, slow and unstable. So, I've been patient and as soon as Google released a Mac OS version of its navigator Chrome I gave it a try. What a charm ! It is fast, stable, works intuitively and on my small 13' MacBook screen optimizes the space taken by the webpage.
Since a couple of months it is also highly customizable with the 'extensions' capability. 
I'll soon report here about a Chrome extension I'm writting for bibliography search based on JournArch.
Before then I just want to share simple webapp crx files I packaged for my personal use (see here for a bunch of them). 
Download zip files and click on crx files.

About Matlab figure export in pdf/eps

posted Sep 22, 2010, 1:51 AM by G. Maze ‎(admin)‎   [ updated Feb 13, 2013, 5:33 AM by Guillaume Maze ]

I export my Matlab figures in the
pdf format and this produces a nice a4 or usletter size pdf document with my plot. 

In fact, Matlab creates a pdf of the size indicated in the 'papertype' or 'papersize' figure properties. 
Most of the time I don't care because my plots fit in a landscape or portrait a4/usletter page.

But when comes the time to write a clean article or a report and embed my figure in a LaTeX document, I want to get rid of all the white space around the plot and have a nice pdf file fitting the plot.

You probably noticed that exporting the figure in the eps format will produce an eps file fitting the plot without extra white spaces around it. 
With my Mac, opening the eps file (with Preview) creates a pdf file exactly as I want it (without white space) and then I just have to "Save as" the file and I'm done.

But there's a more automatic way to do it, without relying on the Mac OS Preview software to produce directly the pdf file.
Let's say your file name (without the file type extension) is : 'figure_output'
Under Matlab, you can type the following:
filo = 'img/figure_output'; 
print(gcf,'-depsc2',sprintf('%s.eps',filo));
system(sprintf('ps2pdf -dEPSCrop %s.eps %s.pdf',filo,filo));
This will create the eps file with Matlab and the pdf file with the system command 'ps2pdf'.

I'm reporting this little trick here because I've been looking for the '-dEPSCrop' option to ps2pdf for a while now ! 
Without this option the pdf file is of the default paper size of the ps2pdf command.

On a daily basis I use three Matlab functions to manage my figures.
- figure_land/figure_tall: to set the figure's orientation and paper size correctly (this correspond to setting A4 landscape or portrait formats). It also modifies the figure's size and color, so that it looks more like what is expected.
- exportp: this function allows one to simply manage figure export to pdf. Simply put it the figure's handle, a file name (no extension) and an orientation. This allows to export clean pdf to portrait, landscape and 'crop' formats



GFD Search Engine (add to Firefox custom search)

posted Apr 6, 2010, 12:53 AM by G. Maze ‎(admin)‎

I started aggregating a bunch of useful sites to look for GFD informations on the web, some kind of specialized Google search engine.
You can find its homepage here:
http://www.google.com/cse/home?cx=006346700993241352089:qyczjxt9rgc
or here:
http://www.ifremer.fr/lpo/files/gmaze/gfd_search.html

If you want to insert it on your webpage, visit this page, or if you want to add it to your Google home page click on this button:
But if you want to add it to your Firefox Custom Search Engines, visit this page (which will reveal the engine parameters) and simply try to change the search engine on your Firefox Search toolbar, you should see an Add "GFD Search" line, click on it and you're all set !
The xml file is here.


Online access to articles with your library and proxy (bookmarklet to modify url)

posted Mar 23, 2010, 3:01 AM by G. Maze ‎(admin)‎   [ updated Sep 23, 2010, 3:11 AM ]

MIT provides a very nice method to access articles from publishers. It's through their library proxy. I used to use a simple trick, simply adding .libproxy.mit.edu at the end of the article url hostname. For example I changed:
to:

But I get tired of constantly adding this extension when needed, so I created a very simple javascript which does the job for me by a simple click. The script splits the window.location variable and add .libproxy.mit.edu where it's needed.
I wish I could simply add a link on this page you would have to drag to your bookmark bar and click it when you want, but google apps doesn't allow it ;-(. 
Anyway, open your bookmark manager, create a new bookmark you'll probably place in your bookmark bar and in the bookmark url insert the code below:
javascript:(function(){var newurl = window.location.protocol+'//'+window.location.host+'.libproxy.mit.edu'+window.location.pathname+window.location.search;window.location=newurl;})();

It should work fine (at least it does for me, on ams, agu and science direct so far)

ps: for those with access to the Biblioplanet, you can use:
javascript:(function(){var newurl = window.location.protocol+'//'+window.location.host+'.biblioplanets.gate.inist.fr'+window.location.pathname+window.location.search;window.location=newurl;})();

ps: All of this is now explained on the MIT library webpage here:

Update:
My javascript is not the shortest one, MIT script is as follow in the bookmarlet: void(location.hostname = location.hostname + '.libproxy.mit.edu');

1-10 of 18