python - Traversing multiple dataframes simultaneously -
i have 3 dataframes of 3 users same column names time, compass data,accelerometer data, gyroscope data , camera panning information. want traverse dataframes simultaneously check particular time user has performed camera panning , return user(like in data frame panning has been detected particular time). have tried using dash achieving parallelism in vain. below code
import pandas pd import glob import numpy np import math scipy.signal import butter, lfilter order=3 fs=30 cutoff=4.0 data=[] gx=[] gy=[] g_x2=[] g_y2=[] datalist = glob.glob(r'c:\users\chaitanya\desktop\thesis\*.csv') csv in datalist: data.append(pd.read_csv(csv)) in range(0, len(data)): data[i] = data[i].groupby("time").agg(lambda x: x.value_counts().index[0]) data[i].reset_index(level=0, inplace=true) def butter_lowpass(cutoff,fs,order=5): nyq=0.5 * fs nor=cutoff / nyq b,a=butter(order,nor,btype='low', analog=false) return b,a def lowpass_filter(data,cutoff,fs,order=5): b,a=butter_lowpass(cutoff,fs,order=order) y=lfilter(b,a,data) return y in range(0,len(data)): gx.append(lowpass_filter(data[i]["gyro_x"],cutoff,fs,order)) gy.append(lowpass_filter(data[i]["gyro_y"],cutoff,fs,order)) g_x2.append(gx[i]*gx[i]) g_y2.append(gy[i]*gy[i]) g_rad=[[] _ in range(len(data))] g_ang=[[] _ in range(len(data))] in range(0,len(data)): j in range(0,len(data[i])): g_ang[i].append(math.degrees(math.atan(gy[i][j]/gx[i][j]))) data[i]["ang"]=g_ang[i] panning=[[] _ in range(len(data))] in range(0,len(data)): j in data[i]["ang"]: if 0-30<=j<=0+30: panning[i].append("panning") elif 180-30<=j<=180+30: panning[i].append("left") else: panning[i].append("none") data[i]["panning"]=panning[i] result=[[] _ in range(len(data))] in range (0,len(data)): result[i].append(data[i].loc[data[i]['panning']=='panning','ang'])
i'm going make assumption want traverse simultaneously in time. in case, want 3 dataframes have index in dimension want traverse.
i'll generate 3 dataframes rows representing random seconds in 9 second period.
then, i'll align these pd.concat
, ffill
able reference last known data gaps.
seconds = pd.date_range('2016-08-31', periods=10, freq='s') n = 6 ssec = seconds.to_series() sidx = ssec.sample(n).index df1 = pd.dataframe(np.random.randint(1, 10, (n, 3)), ssec.sample(n).index.sort_values(), ['compass', 'accel', 'gyro']) df2 = pd.dataframe(np.random.randint(1, 10, (n, 3)), ssec.sample(n).index.sort_values(), ['compass', 'accel', 'gyro']) df3 = pd.dataframe(np.random.randint(1, 10, (n, 3)), ssec.sample(n).index.sort_values(), ['compass', 'accel', 'gyro']) df4 = pd.concat([df1, df2, df3], axis=1, keys=['df1', 'df2', 'df3']).ffill() df4
you can proceed walk through via iterrows()
for tstamp, row in df4.iterrows(): print tstamp
Comments
Post a Comment