Comparison with R / R libraries

Since pandas aims to provide a lot of the data manipulation and analysis functionality that people use R for, this page was started to provide a more detailed look at the R language and its many third party libraries as they relate to pandas. In comparisons with R and CRAN libraries, we care about the following things:

• Functionality / flexibility: what can/cannot be done with each tool
• Performance: how fast are operations. Hard numbers/benchmarks are preferable
• Ease-of-use: Is one tool easier/harder to use (you may have to be the judge of this, given side-by-side code comparisons)

This page is also here to offer a bit of a translation guide for users of these R packages.

For transfer of DataFrame objects from pandas to R, one option is to use HDF5 files, see External Compatibility for an example.

Quick Reference

We?ll start off with a quick reference guide pairing some common R operations using dplyr with pandas equivalents.

Querying, Filtering, Sampling

R	pandas
dim(df)	df.shape
head(df)	df.head()
slice(df, 1:10)	df.iloc[:9]
filter(df, col1 == 1, col2 == 1)	df.query('col1 == 1 & col2 == 1')
df[df$col1 == 1 & df$col2 == 1,]	df[(df.col1 == 1) & (df.col2 == 1)]
select(df, col1, col2)	df[['col1', 'col2']]
select(df, col1:col3)	df.loc[:, 'col1':'col3']
select(df, -(col1:col3))	df.drop(cols_to_drop, axis=1) but see [1]
distinct(select(df, col1))	df[['col1']].drop_duplicates()
distinct(select(df, col1, col2))	df[['col1', 'col2']].drop_duplicates()
sample_n(df, 10)	df.sample(n=10)
sample_frac(df, 0.01)	df.sample(frac=0.01)

[1]	R?s shorthand for a subrange of columns (select(df, col1:col3)) can be approached cleanly in pandas, if you have the list of columns, for example df[cols[1:3]] or df.drop(cols[1:3]), but doing this by column name is a bit messy.

Sorting

R	pandas
arrange(df, col1, col2)	df.sort_values(['col1', 'col2'])
arrange(df, desc(col1))	df.sort_values('col1', ascending=False)

Transforming

R	pandas
select(df, col_one = col1)	df.rename(columns={'col1': 'col_one'})['col_one']
rename(df, col_one = col1)	df.rename(columns={'col1': 'col_one'})
mutate(df, c=a-b)	df.assign(c=df.a-df.b)

Grouping and Summarizing

R	pandas
summary(df)	df.describe()
gdf <- group_by(df, col1)	gdf = df.groupby('col1')
summarise(gdf, avg=mean(col1, na.rm=TRUE))	df.groupby('col1').agg({'col1': 'mean'})
summarise(gdf, total=sum(col1))	df.groupby('col1').sum()

Base R

Slicing with R?s c

R makes it easy to access data.frame columns by name

or by integer location

Selecting multiple columns by name in pandas is straightforward

Selecting multiple noncontiguous columns by integer location can be achieved with a combination of the iloc indexer attribute and numpy.r_.

aggregate

In R you may want to split data into subsets and compute the mean for each. Using a data.frame called df and splitting it into groups by1 and by2:

The groupby() method is similar to base R aggregate function.

For more details and examples see the groupby documentation.

match / %in%

A common way to select data in R is using %in% which is defined using the function match. The operator %in% is used to return a logical vector indicating if there is a match or not:

The isin() method is similar to R %in% operator:

The match function returns a vector of the positions of matches of its first argument in its second:

The apply() method can be used to replicate this:

For more details and examples see the reshaping documentation.

tapply

tapply is similar to aggregate, but data can be in a ragged array, since the subclass sizes are possibly irregular. Using a data.frame called baseball, and retrieving information based on the array team:

In pandas we may use pivot_table() method to handle this:

For more details and examples see the reshaping documentation.

subset

The query() method is similar to the base R subset function. In R you might want to get the rows of a data.frame where one column?s values are less than another column?s values:

In pandas, there are a few ways to perform subsetting. You can use query() or pass an expression as if it were an index/slice as well as standard boolean indexing:

For more details and examples see the query documentation.

with

An expression using a data.frame called df in R with the columns a and b would be evaluated using with like so:

In pandas the equivalent expression, using the eval() method, would be:

In certain cases eval() will be much faster than evaluation in pure Python. For more details and examples see the eval documentation.

plyr

plyr is an R library for the split-apply-combine strategy for data analysis. The functions revolve around three data structures in R, a for arrays, l for lists, and d for data.frame. The table below shows how these data structures could be mapped in Python.

R	Python
array	list
lists	dictionary or list of objects
data.frame	dataframe

ddply

An expression using a data.frame called df in R where you want to summarize x by month:

In pandas the equivalent expression, using the groupby() method, would be:

For more details and examples see the groupby documentation.

reshape / reshape2

melt.array

An expression using a 3 dimensional array called a in R where you want to melt it into a data.frame:

In Python, since a is a list, you can simply use list comprehension.

melt.list

An expression using a list called a in R where you want to melt it into a data.frame:

In Python, this list would be a list of tuples, so DataFrame() method would convert it to a dataframe as required.

For more details and examples see the Into to Data Structures documentation.

melt.data.frame

An expression using a data.frame called cheese in R where you want to reshape the data.frame:

In Python, the melt() method is the R equivalent:

For more details and examples see the reshaping documentation.

cast

In R acast is an expression using a data.frame called df in R to cast into a higher dimensional array:

In Python the best way is to make use of pivot_table():

Similarly for dcast which uses a data.frame called df in R to aggregate information based on Animal and FeedType:

Python can approach this in two different ways. Firstly, similar to above using pivot_table():

The second approach is to use the groupby() method:

For more details and examples see the reshaping documentation or the groupby documentation.

factor

pandas has a data type for categorical data.

In pandas this is accomplished with pd.cut and astype("category"):

For more details and examples see categorical introduction and the API documentation. There is also a documentation regarding the differences to R?s factor.