Need to explain the kdb/q script to save partitioned table

Question

I'm trying to understand this snippet code from:

https://code.kx.com/q/kb/loading-from-large-files/

to customize it by myself (e.x partition by hours, minutes, number of ticks,...):

$ cat fs.q
\d .Q

/ extension of .Q.dpft to separate table name & data
/  and allow append or overwrite
/  pass table data in t, table name in n, : or , in g
k)dpfgnt:{[d;p;f;g;n;t]if[~&/qm'r:+en[d]t;'`unmappable];
 {[d;g;t;i;x]@[d;x;g;t[x]i]}[d:par[d;p;n];g;r;<r f]'!r;
 @[;f;`p#]@[d;`.d;:;f,r@&~f=r:!r];n}

/ generalization of .Q.dpfnt to auto-partition and save a multi-partition table
/  pass table data in t, table name in n, name of column to partition on in c
k)dcfgnt:{[d;c;f;g;n;t]*p dpfgnt[d;;f;g;n]'?[t;;0b;()]',:'(=;c;)'p:?[;();();c]?[t;();1b;(,c)!,c]}

\d .

r:flip`date`open`high`low`close`volume`sym!("DFFFFIS";",")0:
w:.Q.dcfgnt[`:db;`date;`sym;,;`stats]
.Q.fs[w r@]`:file.csv

But I couldn't find any resources to give me detail explain. For example:

if[~&/qm'r:+en[d]t;'`unmappable];

what does it do with the parameter d?

Are you familiar with the k programming language underlying q/kdb? Most of those snippets require knowledge of k code, as well as the contents of the .Q namespace (.e.g the "en" is .Q.en which is documented) — terrylynch
– terrylynch, Commented Mar 15, 2020 at 20:38
I'm learning q, I know that it's in Q space, .Q.en is used to enumerate varchar. And if[~&/qm'r:+en[d]t;'unmappable];, can raise unmappable error. But in total, i coundn't it. — Duc Tran
– Duc Tran, Commented Mar 16, 2020 at 1:12
Ok. As I said, you need to be familiar with k code (specifically k4) to understand all of it. + is flip, & is where, ~ is not etc etc. Using this you can rewrite the functions in q terms and that should help you understand — terrylynch
– terrylynch, Commented Mar 16, 2020 at 9:00
@terrylynch. Can you with me the link to see + is flip and & is "where". I didn't find them in code.kx.com/q/ref. Thank you — Duc Tran
– Duc Tran, Commented Mar 18, 2020 at 6:40

terrylynch · Accepted Answer · 2020-03-18 09:19:36Z

1

(Promoting this to an answer as I believe it helps answer the question).

Following on from the comment chain: in order to translate the k code into q code (or simply to understand the k code) you have a few options, none of which are particularly well documented as it defeats the purpose of the q language - to be the wrapper which obscures the k language.

Option 1 is to inspect the built-in functions in the .q namespace

q).q
          | ::
neg       | -:
not       | ~:
null      | ^:
string    | $:
reciprocal| %:
floor     | _:
...

Option 2 is to inspect the q.k script which creates the above namespace (be careful not to edit/change this):

vi $QHOME/q.k

Option 3 is to lookup some of the nuggets of documentation on the code.kx website, for example https://code.kx.com/q/wp/parse-trees/#k4-q-and-qk and https://code.kx.com/q/basics/exposed-infrastructure/#unary-forms

Options 4 is to google search for reference material for other/similar versions of k, for example k2/k3. They tend to be similar-ish.

Final point to note is that in most of these example you'll see a colon (:) after the primitives....this colon is required in q/kdb to use the monadic form of the primitive (most are heavily overloaded) while in k it is not required to explicitly force the monadic form. This is why where will show as &: in the q reference but will usually just be & in actual k code

answered Mar 18, 2020 at 9:19

terrylynch

14k1 gold badge15 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Duc Tran Over a year ago

Thank you, it helps me so much.

Collectives™ on Stack Overflow

Need to explain the kdb/q script to save partitioned table

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related