That's a tall order. Jaco had hands of iron, I suspect his nut was not that low, but I get the drift. The best I can do is tell you about my preferences.
********* Disclaimer **********
What ever work, or modifications, you make to your instrument(s) is yours exclusively. I do not warrant that any measurement or modification that I make to my basses is correct for you or your bass. I am only offering my opinion and experience.
*****************************
In general, Nearly straight fingerboard + Low Nut + Saddle height (just right) = Mwah sound and easy to play. Strings contribute to Mwah too, but it is up to you to find what you like. I use D'Addario EXP170-5SL (coated nickle roundwound, super long).
Nut filing is nearly the last part of setup so I advise that you get experience doing all phases of setup before cutting or filing your first nut.
For relief, read other sources on the subject, get informed on proper neck relief adjustment. I prefer relief as flat as possible for fretless bass. DO NOT FORCE the tension adjustment nut. If your neck will not approach straight without using excessive force do not attempt to over torque it. Read up on using a beam and clamp to put your neck into backbow, or to the straightness you desire, then adjust the tensioning nut. DON'T FORCE IT!!!!! So what is forcing the tensioning nut? Well, I have not measured the torque I use but would venture the information is out there somewhere. The guideline I use is this. If I estimate the torque I am exerting is sufficient to compress wood fibers, then I have used too much force. Also, the tensioning nut tends to bind when excessive force is used. If the nut becomes hard to turn, you have probably reached the limit.
Excessive torque on the tensioning nut can destroy a bass. This should not imped you from adjusting your neck, just become informed, and use care.
Nut filing:
Tools needed for nut filing:
Feeler gauges
Nut files for your string gauges.
Micrometer, to confirm files match strings, or are close.
If you screw up and need to cut a new nut more tools are needed.
When filing a nut slot, it is easily possible to take it too low. The right way to repair a string slot cut too low is to make an entirely new nut. However it is possible to build up the too low string slot and re-file it. Such a repair may not sound as good as a nut with full integrity but it would save the individual some work. Advice on luthier technique may be found on other sites such as Stewart-McDonald
http://www.stewmac.com/nuts Read this or other authoritative information.
When filing a nut you need nut files. If you do not have the right files the bottom of the string slot will probably not be the correct diameter (gauge) for your string. String gauge in North America is usually measured in 1/1000 of inches (such as .090).
As far as I know, there is no proper specification for nut height over fingerboard. There is a minimum, and then there is everything above that minimum. The minimum differs with each instrument unless your fingerboard is absolutely perfectly straight. Very few instrument achieve, or approximate, this level of precision. Graphite/Phenolic necks such as Modulus and ZON come close, and that is what I use. My main (mag P/U) fretless is a 1988 (pre tension rod) Modulus Q5 (wide) that is rock stable and darn near perfect in straightness with my preferred strings, tuned.
Before attempting to file the nut, your neck needs to be adjusted for the relief you prefer. I like my fingerboard as flat as is possible. AND, the bridge needs to be set properly. THEN one can move on to final nut filing.
I would suggest that you get the nut close but not final. Do a check of setup in general, readjust setup, then proceed.
Keep in mind that good setup may not be achieved in one work session. Your ultimate setup might be the result of months of occasional tweaking. It could be said that setup is a recursive endeavour. On the other hand, a professional luthier who does many fretless jobs can probably get very close to ideal action in one work session. Then the weather changes and your bass may absorb humidity changing things necessitating a new set up.
With that said, my preference for string height at nut is "as low as practical", or near minimum but not actually that low. The only way to get this correct is by trial. File a thousandth, tune it up and test. File another thousandth, etc. when it plays great, sings, mwahs, and you end up playing longer than anticipated, STOP filing.
The gap between the fingerboard and strings, at the nut, on my Modulus Q5 varies with each string.
G = .014in .356mm (about the thickness of a business card)
D = .020in .508mm
A = .020in .508mm
E = .022in .559mm
B = .029in .737mm (two business cards)
If your neck has problems such as an "S" bend, a high spot, or other imperfections, these measurements will probably be too low.
These measurements do not represent the minimum, rather is where I like it. The bass Mwahs like a cat. I find this action easy to play in the first position. Since string oscillation is greater on the big strings, it follows that the gap between the fingerboard and the bottom of each string must also be successively greater.
with a capo on the 1st and holding down the 12th what is the distance from the 7th to the string?
Since my neck is near flat, this measurement is not meaningful. See comment above regarding relief. I presume that you are interested in action height over the fingerboard. I adjust string height at each saddle measuring the gap at the highest note, or end (highest pitch, or position) of the fingerboard using a self-made white plastic tapered gauge with a mark (in ink) for my preferred action height. Move the wedge shaped (tapered) measuring device between the string and fingerboard at the highest position until contact is made. Mark using pencil, measure at the mark using a micrometer. Adjust bridge saddles until satisfied. Mark your preferred action height with ink for future reference.
On my basses this marked measurement is .085in 2.159mm for the G string. As with the nut measurement, it gets greater with larger strings. this simple device makes action setup after changing string types a snap.
Pick-up height:
Adjusting P/U height on most basses involves turning Phillips head screws to raise or lower. Be sure to use the correct size screwdriver. Some basses such as Alembics have a more complicated system but the theory of operation remains the same for all magnetic P/Us.
Several factors are involved. Are you going to be slapping this instrument? If so, you will want your P/Us a bit lower to avoid string P/U contact. Do you have active electronics? If yes then having them high for greater output is less important. Do you have heavy right hand technique? If yes then you will want the P/Us lower as with slappers.
If your right hand technique is not heavy, and you do not slap, then you will probably want your P/Us as high as is practical for better output level. The output of magnetic P/Us diminishes with the square of the distance from the inductor (oscillating string). Meaning that the difference in output between 1/8" inch distance and 1/4" is profound. It is important to adjust your P/Us to be as close to the strings as possible but far away enough that string/P/U contact is unlikely.
In my experience, P/U height does not effect tone (except when the strings hit them) but is very important to getting good output level and quality (S/N ratio), especially for passive instruments. Some may argue that there is a proximity effect emphasizing bass frequencies, but I have not found this to be true for active basses.
If you have two P/Us, the next consideration is balance between P/Us. Jaco (according to interview) seldom used his neck P/U, relying almost exclusively on the bridge P/U for his tone. So it is generally accepted that the bridge P/U is more important for fretless tone.
Most basses these days have a P/U pan control with a center detent, or a P/U selector switch with a combined (center) position that is equivalent to the pan center detent. This center position is important for gigging musicians because it can be found by touch and represents a preset tone reference point. Thus this position needs to be as hip a tone as possible. You can bias (means emphasize one P/U over the other) your combined P/U tone at center position by raising or lowering your pickups thereby changing the loudness of each.
On my fretless (mentioned above) the neck P/U is naturally louder due to greater string movement/energy so I lowered it to relatively increase the sound of the more desirable (in tone) bridge P/U. Here is how I do it. Select the bridge P/U, raise the bridge P/U as high as your technique will allow AND avoiding string to P/U contact during your most vigorous playing. If your strings hit the P/U then lower it until contact ceases.
Still on the bridge P/U. String to string loudness also needs to be adjusted. Listen to each string while uniformly playing each in succession. If the higher strings are louder then that side needs to be lowered slightly. Conversely, if the lower string are louder, that side needs to be lowered slightly. Adjust until all strings are about uniform in output. If you want to get technical, plug-in to a recorder and watch the VU meters. Some P/Us have individual pole piece adjustments, with these, use the previous for rough adjustment and turn the pole pieces for fine adjust.
Once you have determined your preferred bridge P/U height, capo (or hold down the string) at the highest position (equivalent to highest fret) and measure the gap over P/U for future reference.
Now move the P/U pan/selector to exclusively either 100% bridge, or 100% neck P/U. Move between the two and note any differences in volume. Usually the neck P/U will be louder if both are of similar winding. Some P/U sets are wound differently to compensate for this natural difference in output.
Next, put your P/U pan/selector in the center/combined position. You are probably hearing more of the neck P/U than bridge. If you have a pan control now is the time to experiment and find the hippest tone that is within 10% of the center detent, this is your bias offset. Make a mental note of this sound, or actually record it. I assume that your bias is towards the bridge P/U but it does not have to be that way. This tone is the one to attempt to replicate at the center detent by lowering (or raising (not likely)) the neck P/U.
Now set the P/U pan/selector in the center/combined position. Listen to how this sound differs from your bias offset tone. Lower the neck P/U until the sound comes closer to your bias offset tone. Often it is not possible to exactly match that sound, just get close as is possible without screwing the neck pickup so low that it is pinched against the bottom of the pickup route.
If you have a P/U selector switch and not a pan control the procedure is similar but you do not have the luxury of experimenting with a pan pot. Just trust that it is the neck P/U that needs to go lower (for fretless bass). Lower the neck P/U in the combined position until you like the sound.
Repeat the string to string balance for the neck P/U. If the P/U is bottomed out you will have to raise the side with lower output.
Best of luck to you performing set-up on your basses. There is more to it than is mentioned in this post. I hope this answers your questions.
Set-up is a DIY affair providing you have the correct tools and information.