In our instance, I filter a reference field based on another reference field by doing that following:

You could use an advanced Reference Qualifier on the reference field.  Here is our example:

"SOW Number" - Current reference field I am updating the Reference Qual on

"current.variables.customer" - form field reference to custom table

"u_client" - reference field on my custom table

Below is my Advanced Reference Qualifier on the "SOW Number" field.

javascript:'u_client='+current.variables.customer+'^u_active=true';

So basically this filters the "SOW Number" reference field by the "Customer" field which is also a reference field.  It only shows SOW numbers associated with the customer chosen that are active.