Answer by Anton Protopopov for Equivalent for str.join in pandas
If you don't mind about space at the end of your rows you could use sum which is a bit faster then manually typing df[0] +''+ df[1] +''+ df[2]:In [25]: (df +'').sum(axis=1)Out[25]:0 word1 word2...
View ArticleHow to show date and time on x axis
I would like to assign for x axis in matplotlib plot full date with time but with autoscale I could get only times or dates but not both.import matplotlib.pyplot as pltimport pandas as pdtimes =...
View ArticleAnswer by Anton Protopopov for How to find duplicates in pandas?
You could use duplicated for that:df[df.duplicated()]You could specify keep argument for what you want, from docs:keep : {‘first’, ‘last’, False}, default ‘first’first : Mark duplicates as True except...
View ArticleAnswer by Anton Protopopov for Format datetime in pandas dataframe using...
You can use .dt.strftime attribute after converting to timestamp series. The following command should do the trick:In [26]: pd.to_datetime(df[0]).dt.strftime('%H:%M:%S')Out[26]:0 08:14:241...
View ArticleAnswer by Anton Protopopov for Making Python loggers output all messages to...
You could create two handlers for file and stdout and then create one logger with handlers argument to basicConfig. It could be useful if you have the same log_level and format output for both...
View ArticleAnswer by Anton Protopopov for Splitting timestamp column into separate date...
The easiest way is to use the pandas.Seriesdt accessor, which works on columns with a datetime dtype (see pd.to_datetime). For this case, pd.date_range creates an example column with a datetime dtype,...
View ArticleAnswer by Anton Protopopov for Line color depending on variable
As I said you could find the answer from the link I attached in the comment:Dates = ['01/01/2014', '02/01/2014', '03/01/2014', '03/01/2014', '04/01/2014', '05/01/2014']Values = [3, 4, 6, 6, 5, 4]Colors...
View ArticleHow to get current date and time from GPS unsegment time in python
I have gps unsegmented time like this:Tgps = 1092121243.0And I'd like to understand what date and time is that. The begining of GPS time is 6 January 1980. Python function datetime.utcfromtimestamp...
View ArticleAnswer by Anton Protopopov for Get a list from Pandas DataFrame column headers
It's interesting, but df.columns.values.tolist() is almost three times faster than df.columns.tolist(), but I thought that they were the same:In [97]: %timeit df.columns.values.tolist()100000 loops,...
View ArticleAnswer by Anton Protopopov for How can I make pandas dataframe column headers...
You could do it easily with str.lower for columns:df.columns = df.columns.str.lower()Example:In [63]: dfOut[63]: country country isocode year XRAT tcgdp0 Canada CAN 2001 1.54876 9.249094e+051 Canada...
View ArticleAnswer by Anton Protopopov for Combine two JSON dictionaries in Python?
IIUC you need to join two dicts into one, you could do it with update:a = {"cars": 1, "houses": 2, "schools": 3, "stores": 4}b = {"Pens": 1, "Pencils": 2, "Paper": 3}a.update(b)print(a)output would...
View Articletimedelta error with numpy.longdouble dtype
I have times with dtype numpy.longdouble and when I'm trying to use that values with timedelta function I've got errors. But when I convert it to numpy.float64 everything is fine. Could somebody...
View ArticleAnswer by Anton Protopopov for Split one level of an multi index into columns
You could use unstack for that:df2.unstack(level='a')Example:In [146]: df2.unstack(level='a')Out[146]: ca bar foobbuzz 8 7fizz 6 3After that you'll get multiindexed columns. If you need to get flat...
View ArticleAnswer by Anton Protopopov for Create new folder with pathlib and write files...
You can directly initialize filepath and create parent directories for parent attribute:from pathlib import Pathfilepath = Path("temp/test.txt")filepath.parent.mkdir(parents=True, exist_ok=True)with...
View ArticleAnswer by Anton Protopopov for Prevent pandas from interpreting 'NA' as NaN...
You could use parameters keep_default_na and na_values to set all NA values by hand docs:import pandas as pdfrom io import StringIOdata = """PDB CHAIN SP_PRIMARY RES_BEG RES_END PDB_BEG PDB_END SP_BEG...
View ArticleAnswer by Anton Protopopov for Use of loc to update a dataframe python pandas
As you could see from the warning you should use loc[row_index, col_index]. When you subsetting your data you get index values. You just need to pass for row_index and then with comma...
View ArticleNumexpr not found in pandas
I install numexpr package via pip on my Windows 7 machine:pip list | grep numexprnumexpr (2.4.6)but when I open ipython and trying to use df.query it shows an error:ImportError: 'numexpr' not found....
View ArticleWhy does apply change dtype in pandas dataframe columns
I have the following dataframe:import pandas as pdimport numpy as npdf = pd.DataFrame(dict(A = np.arange(3), B = np.random.randn(3), C = ['foo','bar','bah'], D = pd.Timestamp('20130101')))print(df) A B...
View ArticleAnswer by Anton Protopopov for Pandas: Converting to numeric, creating NaNs...
In pandas 0.17.0convert_objects raises a warning:FutureWarning: convert_objects is deprecated. Use the data-type specific converters pd.to_datetime, pd.to_timedelta and pd.to_numeric.You could use...
View ArticleAnswer by Anton Protopopov for Remove non-numeric rows in one column with pandas
You could use standard method of strings isnumeric and apply it to each value in your id column:import pandas as pdfrom io import StringIOdata = """id,name1,A2,B3,Ctt,D4,E5,Fde,G"""df =...
View Article