2,525 questions
0
votes
1
answer
30
views
Compare table meta across servers ignoring column order in KDB
I need to compare the schema (meta) of a table that exists on two different servers in kdb+. On Server A, the table has columns in order col1; col2; col3, and on Server B, the same table has columns ...
0
votes
1
answer
72
views
How to create nested dict from keyed table?
I have a keyed table, like
([k1: 20?`a`b`c; k2: 20?`x`y`z; k3: 20?1000] ; v: 20?`A`B`C)
I need to pass this on to another application, which only accepts nested dicts.
How can I turn this table into ...
- 4,265
0
votes
1
answer
74
views
How do I transpose a table and store it as a table? [duplicate]
I have a table with data as below:
t:([]enlist val10s:4.44;enlist val20s:4.18;enlist val30s:3.51;enlist val40s:3.29;enlist val50s:5.87;enlist val60s:7.36);
I would like to transpose this table and ...
- 139
1
vote
2
answers
68
views
How to add two cols of different kind and convert to sym?
I have a table with 2 cols, name and id, and I am trying to add a new col which is a combination of name and id with a . in between.
tab:([]name:`aa`bb`cc;id:1,2,3)
tab:update info:`$(string name,' ...
- 407
1
vote
1
answer
142
views
KDB PYKX how to enable parallel reads (and still allow writes)?
I’m building a FastAPI service that talks to kdb+/q via PyKX. Currently it serves endpoints that run queries through Embedded Q on an inmemory global variable PYKX Table. I am now scaling my ...
- 93
0
votes
3
answers
128
views
Recursively mod the floor division of a list in kdb
How can we recursively mod the floor division of a number in kdb?
Eg. get base 62 remainders of a number 11157.
// input 11157
// expected output in res variable - 59 55 2
// Logic - Mod x by 62 and ...
- 6,093
1
vote
1
answer
92
views
inconsistent response from q console for kdb q
i am to understand this situation in q.
when I insert simple calculation like below, sometime it returns 4, sometime it return broken pipe. Can someone help explains?
4 +/ [1 2 3 4 5]
enter image ...
- 13
2
votes
1
answer
107
views
In the q/kdb+ TickerPlant example tick.q, is there a reason the tick function uses l::ld d but the endofday uses l::0(`.u.ld;d)?
The q/kdb+ official documentation on the Tickerplant at https://code.kx.com/q/architecture/tickq/#uupd
References the file tick.q found here: https://github.com/KxSystems/kdb-tick/blob/master/tick.q
I ...
- 487
-1
votes
1
answer
83
views
How to find creation date of a file using KDB q [closed]
A job runs and creates a csv report. I want to find creation date using kdb q and want to verify whether its created today.
I have tried .Q.f but it did not work. Also tried system"ls -l fileName&...
0
votes
1
answer
89
views
Is there a standard way to do bounds checking when using `take`? [closed]
What is the best way to implement a non-circular take (#) i.e. take[5; 1 2 3] returning 1 2 3 rather than 1 2 3 1 2?
Would {(x&count y)#y} be the general best? Are there any types of y for which I ...
- 487
0
votes
1
answer
98
views
Getting some columns as raw data while others converted to pandas types
Is there a way in KDB/pykx to get only some columns as raw data while get others converted to pandas types?
In the example below, I want to be able to do what is shown in the last line (for variable ...
- 2,855
1
vote
2
answers
94
views
how to join based on a range of dates and a name?
I want to join tables on two columns date and tradename but instead of the tables matching exactly for the two columns like for a left join. Only the tradename needs to be an exact match and the date ...
- 51
1
vote
1
answer
99
views
in kdb how do you save a 2d matrix into a binary format such that it can be read in python using numpy.fromfile or numpy.memmap?
My goal is to save a huge 2d matrix as binary in kdb q, so that my python code can import it as numpy matrix. I played with numpy.fromfile and numpy.memmap but I don't think kdb is writing it into a ...
- 33
0
votes
0
answers
89
views
upsert but records that match on key are not updated
q)show x:`a`b xkey ([] a:2#`p; b:`x`y; c:1 2);
a b| c
---| -
p x| 1
p y| 2
q)show y:update c:(3;4) from x;
a b| c
---| -
p x| 3
p y| 4
q)x upsert y
a b| c
---| -
p x| 3
p y| 4
The documentation for ...
- 934
1
vote
2
answers
87
views
How do you run system commands like loading a db over a handle
I've connected to a process and I want to load a db into it from another process. How do you do this in kdb+ as you can't use system"" because it clashes with handle""
- 51
1
vote
1
answer
147
views
KDB boolean null value
I am using the KDB C Api to write data into few KDB tables.
There is the need to insert in our tables null values.
I found all the null values I need for different types except for boolean.
Is there ...
0
votes
1
answer
206
views
How to update a string column in kdb+/q?
In kdb+/q a string is essentially a list/vector. I have a table which normally is partitioned but for now I'm doing it all in memory as a tester.
There is a column I wish to update, which already is a ...
- 25
1
vote
2
answers
88
views
Decrement next n rows of a column based on condition
How can I create an update for column b which decrements b by one for the next 2 rows following a 3 value of b?
It can either be a rewrite of the existing update or via a separate update.
q)t: ([] a: ...
- 341
1
vote
2
answers
120
views
How to sum quantities by symbol, side, and price, grouping timestamps within 1ms of each other?
I have a table t with columns: symbol, side, price, time, and qty.
I want to sum qty by symbol, side, and price, but also group rows where time values are ≤1ms apart.
How can I achieve this?
for ...
- 11
0
votes
2
answers
67
views
namespace forced first null key-value pair
Why does q force a first null key-value into a namespace? What purpose does it serve?
q).thespace.thing: 42
q).thespace
| ::
thing| 42
- 341
0
votes
1
answer
63
views
PYKX not allowing to change Python attribute
I am trying to do the following:
import pykx
old=":host:Port:user:old"
pykx.q.my_handle=old
new=":host:Port:user:new"
pykx.q.my_handle=new
The above works as expected. If I then ...
- 93
0
votes
0
answers
87
views
empty table in schema causes 'type error in .Q.hdpf function
I have a schema with several tables, usually they all contain data. I have eod function , which calls Q.hdpf to save tables from rt. Everything works as expected. However, when one of the tables is ...
- 2,632
-3
votes
1
answer
131
views
KDB + DB Maintenance
can any one advise how I would run a db Maintenance step on a hdb to update
a table where column called "true" holds boolean values 1b 0b.
I want to update to Y where column true =1b and N ...
- 19
1
vote
0
answers
115
views
How to compare trades for matching uid and sym across dates comparing performance?
Custom trades table for a given day below:
date sym uid deviation price
-------------------------------------------
2021.10.12 793640 207280 3.247719 60.32779
2021.10.12 793640 107701 1....
- 51
-1
votes
2
answers
93
views
How to add columns with next N values?
date sym uid deviation
-----------------------------------
2021.07.10 788779 107701 2.900847
2021.08.21 791540 107701 0.6709649
2021.09.11 790922 107701 0.326201
2021.09.22 791791 ...
- 51
0
votes
2
answers
74
views
Ratio of numerical columns based on the values of categorical column in kdb
I have a table as below for which I have to determine the buySellRatio of notional based on the side grouped by ratings and dt.
// Sample Input -
dt ratings side notional
---------------------...
- 6,093
-1
votes
2
answers
90
views
Update Date column in a table in kdb+/q
How can I change the table Date Column to date format in kdb(example: 16/8/2022 to 2022.08.16)?
Input:
Stock
Date
Open
High
Low
Close
Adj Close
Volume
AAPL
16/8/2022
172.78
173.71
171.66
173.03
172....
- 31
1
vote
1
answer
144
views
Kdb mixed list type
I stumbled upon an interesting bug/feature of .Q.ty.
From the docs (https://code.kx.com/q/ref/dotq/#ty-type), it states upper case for a list of uniform type.
However, I ran into the following bug/...
- 970
0
votes
2
answers
81
views
create a kdb table with price range based on the ticker
While creating a sample testing table, I am trying to generate the marketPrice column values based securityId column, i.e.
securityId=a could have marketPrice between 0-100.
securityId=b could have ...
- 6,093
0
votes
1
answer
74
views
row-wise ema of several columns
In kdb+ let's say I have a table t such that
show t:([] rnk1: 1 2; rnk2: 3 4);
rnk1 rnk2
---------
1 3
2 4
I can calculate the row-wise average avg of the two columns as:
show update r:avg[(...
- 934
-1
votes
1
answer
70
views
Generating a nested functional expression
I was trying to create a functional equivalent via:
q)buildQuery["update rnk:(rnk1+rnk2+rnk3+rnk4)%4 from t"]
"![t;();0b;(enlist`rnk)!enlist (%;(+;`rnk1;(+;`rnk2;(+;`rnk3;`rnk4)));4)]&...
- 934
0
votes
1
answer
92
views
Why is `select count i by date` in a kdb+ hdb gives a rank error?
This query doesn't work
select count i by date from table where date >= .z.d-5
This query works
select count sym by date from table where date >= .z.d-5
Any other operation on that table seems ...
- 49
-3
votes
1
answer
96
views
Adding multiple, parametrized columns using functional update
Given a table t as
ticker time close
---------------------------------------------
IBM 2025.04.07D09:15:00.000000000 22161.6
IBM 2025.04.08D09:15:00.000000000 22535.85
...
- 934
-1
votes
3
answers
102
views
Functional Form in q/KDB+
Give functional form for:
select sum size by ticker:((string[id],'"."),'string[ex])from trade_usq where any not null(size;price)
Here is the table -
trade_usq:([]id:10?`AAPL`GOOG`JPM`MSFT`...
0
votes
1
answer
82
views
How to profile/monitor a KDB tickerplant to trace causes of a slow tickerplant?
I'm trying to use KDB as a low-latency pub/sub message broker that persists all messages in a queryable format.
However, I'm noticing the latency from when the tickerplant receives a message (i.e. ...
- 10.3k
-3
votes
2
answers
174
views
Does this use of aj0 in KDB do a full disk load?
I'm having a bit of mare of a time arguing with my firm's KDB team. They're giving me performance advice that seems to contradict my benchmarks. I'm migrating data into KDB from Postgres to conform ...
- 729
-1
votes
4
answers
284
views
How do I assert a column/list is ordered in KDB
a: 1 2 3 4;
b: 1 2 3 2;
How can I assert that a is sorted and b is not sorted in KDB?
Clarification I mean this algorithm in Q, as in the linear probe with a short circuit.
def is_sorted(xs):
i =...
- 729
0
votes
1
answer
105
views
Efficient iteration over a table for crossable entries KDB+/Q
I have two tables with order information, and am looking for a more efficient way to find whether or not each order in one table has crossing potential with the other.
Example tables:
t1:([]date: 2025....
- 345
2
votes
3
answers
135
views
Return the first missing positive integer
There's a list of integer with both positive and negative values. We need to return the first missing positive integer.
Eg.
Input - 3 -1 1 0 4 , Expected Output - 2
Input - -1 -4 -3, Expected Output -...
- 6,093
0
votes
2
answers
85
views
KDB+ q - What's the logic behind values returned in function as list
I have a simple function which is then defined as a list and invoked as a list:
f:{x*x}
q)(f; neg)[1; 2]
-2
q)(f; neg)[1; 3]
-3
q)(f; neg)[2; 3]
3
q)(f; neg)[4; 3]
3
for the (f; neg)[1; 3] - it ...
- 97
1
vote
2
answers
68
views
Should I create a wrapper around `.z` function when I use them in my app?
In our application, the .z.u function (which returns the username) was originally treated in a case-sensitive manner—so Test2001 and test2001 were considered two distinct users. We've recently decided ...
- 49
0
votes
2
answers
90
views
While condition for iterator to be passed as argument of a function
I have below function for fibonacci which is returning expected output
q)fib:{{x,sum -2#x}/[{last[x] < 100};x]}
q)fib 1 1
1 1 2 3 5 8 13 21 34 55 89 144
How could we replace the value 100 in ...
- 6,093
0
votes
2
answers
91
views
How to append previous 6 trade records as columns for each transaction?
Columns:
date of transaction
uid is the trader id
sym is the trade id
Subset of table:
q) select date, uid, sym from tb
date uid sym
------------------------
2011.08.12 171196 537876
...
- 51
0
votes
1
answer
70
views
Propagating the x1 Column's Data Type to Subsequent Columns in Grouped q Tables
Data:
q) t:([] sym1: 1 2 3 4 5 6 7;
sym2: 1.1 2.2 3.3 4.4 5.5 6.6 7.7;
sym3: 2.1 3.2 4.3 5.4 6.5 7.6 8.7;
sym4: 3.1 4.2 5.3 6.4 7.5 8.6 9.7;
sym5: 4.1 5.2 6.3 7.4 8.5 9.6 10.7;
sym6: 5.1 6.2 7.3 ...
- 51
0
votes
3
answers
73
views
Converting 0N values to the correct null for each data type
So I have 0N values which represents no previous data for that sid, the issue is, is that the 0N is showing up in symbol cols too and I want to convert each case to the null value for the data type it ...
- 51
0
votes
0
answers
134
views
KDB+ Process Not Releasing Memory After Garbage Collection .Q.gc[] and Table Clearing
I am encountering an issue where memory is not being released back to the OS, even after clearing large in-memory tables and running .Q.gc[]. Despite multiple attempts, the process continues to ...
- 21
0
votes
2
answers
87
views
Running (not rolling) correlation by date. "cors", effectively
Is there a way to achieve equivalent functionality of sums except with cor, so that it would effectively be a cors function?
I.e., cor would be calculated for an ever expanding window thru time.
I've ...
- 341
0
votes
1
answer
150
views
how to concatenate list of strings into one long string kdb
I have a list of strings like this
q)select id from temp
id
---------------------------------
,"_"
"01coin" ...
- 2,632
0
votes
1
answer
64
views
kdb update function with iteration
Im trying to have a function where the output becomes the input of the next iteration.
I pass in the table called tab, and want to pass each threshold.
Here is the func:
test:{[data;t]
...
- 19
1
vote
1
answer
113
views
in KDB q, how do you write a custom mavg that drops the top 5 and bottom 5 results before doing moving average
I have a 5min table that has a score value like the table below. How can I add a new column which is the 10-period moving average which drops the highest 2 and lowest 2 scores first, and just average ...
- 33