今天刚刚在看一个关于 SO 的问题时遇到了这个
actions = data.table(User_id = c("Carl","Carl","Carl","Lisa","Moe"),
category = c(1,1,2,2,1),
value= c(10,20,30,40,50))
users = actions[, other_var := 1, by=User_id]
# verbose says: the following is not optimized
users[, value_one := 0 ]
users[actions[category==1], value_one := sum(value), on="User_id", by=.EACHI, verbose=TRUE]
# verbose says: the following is optimized
rbind(
actions[category==1],
unique(actions[,"User_id", with=FALSE])[, value := 0 ],
fill=TRUE)[, sum(value), by=User_id, verbose=TRUE]
对我来说,第一种方式看起来很惯用,考虑到变量最终需要以users
结尾。
另一个: https ://stackoverflow.com/a/47338118/(gtail)
另一个https://stackoverflow.com/a/51569126/应该做DT[, mx := max(pt), by=Subject][, diff := mx - pt][]
我猜
另一个,对内存性能特别感兴趣: https ://stackoverflow.com/q/52189712“data.table 引用语义:遍历所有列的内存使用情况”
另一个,想要scale
/demean 多个变量: https :
另一个使用子集条件按组取最大值并添加 := (参见 akrun 的答案) https://stackoverflow.com/a/54911855/也与#971 的已完成部分有关
只是想强调一下,启用此功能可以有效地将 GForce 用于复杂的表达式,尽管需要一些工作。 例如,我在这篇文章中展示了如何启用它:
slope <- function(x, y) {
x_ux <- x - mean(x)
uy <- mean(y)
sum(x_ux * (y - uy)) / sum(x_ux ^ 2)
}
通过做:
DT <- data.table(grp, x, y)
setkey(DT, grp)
DTsum <- DT[, .(ux=mean(x), uy=mean(y)), keyby=grp]
DT[DTsum, `:=`(x_ux=x - ux, y_uy=y - uy)]
DT[, `:=`(x_ux.y_uy=x_ux * y_uy, x_ux2=x_ux^2)]
DTsum <- DT[, .(x_ux.y_uy=sum(x_ux.y_uy), x_ux2=sum(x_ux2)), keyby=grp]
res.slope.dt2 <- DTsum[, .(grp, V1=x_ux.y_uy / x_ux2)]
而如果:=
支持 GForce,我们可以这样做:
DT <- data.table(grp, x, y)
DT[, `:=`(ux=mean(x), uy=mean(y)), keyby=grp]
DT[, `:=`(x_ux=x - ux, y_uy=y - uy)]
DT[, `:=`(x_ux.y_uy=x_ux * y_uy, x_ux2=x_ux^2)]
DTsum <- DT[, .(x_ux.y_uy=sum(x_ux.y_uy), x_ux2=sum(x_ux2)), keyby=grp]
res.slope.dt3 <- DTsum[, .(grp, x_ux.y_uy/x_ux2)]
这看起来更干净,应该更快。
与@MichaelChirico 的讨论让我意识到这个问题的一个非常亲密的表亲是:
> DT <- data.table(x, y, grp)
> DT[, .(x, mean(x)), keyby=grp]
Detected that j uses these columns: x
Finding groups using forderv ... 1.049s elapsed (0.946s cpu)
Finding group sizes from the positions (can be avoided to save RAM) ... 0.011s elapsed (0.011s cpu)
lapply optimization is on, j unchanged as 'list(x, mean(x))'
GForce is on, left j unchanged
Old mean optimization changed j from 'list(x, mean(x))' to 'list(x, .External(Cfastmean, x, FALSE))'
Making each group and running j (GForce FALSE) ...
collecting discontiguous groups took 1.293s for 999953 groups
eval(j) took 1.860s for 999953 calls
5.517s elapsed (3.862s cpu)
grp x V2
1: 1 0.2151365 0.5512966
2: 1 0.5358256 0.5512966
3: 1 0.8496598 0.5512966
4: 1 0.8480730 0.5512966
5: 1 0.3464458 0.5512966
---
9999996: 1000000 0.2601940 0.5474986
9999997: 1000000 0.7940921 0.5474986
9999998: 1000000 0.3825493 0.5474986
9999999: 1000000 0.1786861 0.5474986
10000000: 1000000 0.9179119 0.5474986
交叉链接到#523。
最有用的评论
只是想强调一下,启用此功能可以有效地将 GForce 用于复杂的表达式,尽管需要一些工作。 例如,我在这篇文章中展示了如何启用它:
通过做:
而如果
:=
支持 GForce,我们可以这样做:这看起来更干净,应该更快。