column values change between loading two partioned tables in KDB (q)

Question

I have two partioned kdb tables on disk (one called trades, one called books). I created the data by using

.Q.dpft[`:I:/check/trades/;2020.01.01;`symTrade;`trades]

and

.Q.dpft[`:I:/check/books/;2020.01.01;`sym;`books]

for each day. If I select data from the trades table and then load the books table (without selecting data) the values in the symTrade columns of my result change to new values. I assume it has got something to do with the paritioning in the books table getting applied to the result from trades table (also the trades table is no longer accessible after loading the books table).

How do I:

keep the trades table accessible after loading the books table?
avoid having my symTrade column overwritten by the sym values in the books table?

Here is an example:

system "l I:/check/trades/";
test: 10 sublist select from trades where date=2020.01.01;
show cols test;
// gives `date`symTrade`time`Price`Qty`Volume
select distinct symTrade from test;
// gives TICKER1
// now loading another table
system "l I:/check/books";
select distinct symTrade from test;
// now gives a different value e.g. TICKER200

user12704155 · Accepted Answer · 2020-07-01 20:50:39Z

4

I think the problem is that you are saving these tables to two different databases.

The first argument in .Q.dpft is the path to the root of the database, and the fourth argument is the name of the table you want to store. So when you do

.Q.dpft[`:I:/check/trades/;2020.01.01;`symTrade;`trades]

You are storing the trades table in a database in I:/check/trades and when you do

.Q.dpft[`:I:/check/books/;2020.01.01;`sym;`books]

you are storing the books table in a database in I:/check/books. I think q can only load in one database at a time, so that might be the problem.

Try doing this

.Q.dpft[`:I:/check/;2020.01.01;`symTrade;`trades]
.Q.dpft[`:I:/check/;2020.01.01;`sym;`books]
system "l I:/check/";

Let us know if that works!

answered Jul 1, 2020 at 20:50

user12704155

Sign up to request clarification or add additional context in comments.

1 Comment

Adam Bonham Over a year ago

To add, the reason your values in the symbol column change is due to the sym file. .Q.dpft will enumerate all symbol values against the sym file, which is stored in the root HDB directory. The sym file is loaded into memory when you load your database. When you load a second database you are overwriting the current sym file with a new one, so your symbol columns in the first table loaded are being mapped against the wrong sym file, and hence displaying incorrect values. Either store the tables in the same database, load the databases into different processes, or unenumerate the first table.

Collectives™ on Stack Overflow

column values change between loading two partioned tables in KDB (q)

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related