This paper was converted on www.awesomepapers.org from LaTeX by an anonymous user.
Want to know more? Visit the Converter page.

Bibliometric Analysis of Senior US Mathematics Faculty

Joshua Paik School of Mathematics and Statistics, University of St Andrews Department of Mathematics, The Pennsylvania State University joshdpaik@gmail.com  and  Igor Rivin Mathematics Department, Temple University Research Department, Edgestream Partners, LLP rivin@temple.edu
Abstract.

We introduce a methodology to analyze citation metrics across fields of Mathematics. We use this methodology to collect and analyze the MathSciNet (http://www.ams.org/mathscinet) profiles of Full Professors of Mathematics at all 131 R1, research oriented US universities. The data recorded was citations, field, and time since first publication. We perform basic analysis and provide a ranking of US math departments, based on age corrected and field adjusted citations.

Key words and phrases:
citations, ranking
1991 Mathematics Subject Classification:
00A99

Introduction

For as long as the authors can remember, there has been discussion of comparable quality of various researchers (in all fields of research, but the authors are most familiar with mathematics, so this paper concerns itself with mathematics exclusively). While such a comparison is not strictly speaking possible (mathematics is not like competitive swimming, where a single number determines the better swimmer), those of us who have been on hiring committees have needed to compare researchers in diverse fields, and those of us who have had students (or job offers) have had to have some sort of estimate of the quality of people independently of age and field (and building on that, to have some reasonably gauge of the quality of departments). The (admittedly ambitious) purpose of this note is to propose an objective metric, based entirely on citations data (as such, it can be gamed, as can any metric be).

Briefly, we normalize the number of MathSciNet citations by dividing it by the number of years since the author’s first paper raised to the magical power 1.31.3. We further segment mathematics into a number of “major fields”, assign mathematicians to fields (this is very difficult for some people, including, ironically, the authors of this paper), and compute the zz-score of the normalized citation number. For each department we then compute the mean zz-score of faculty to compute the department’s ranking.

1. Data

Data was collected between January 16 - January 24, 2020. After accessing the list of R1 schools, we found the faculty lists from the relevant departmental web page, determined their level of seniority, and searched their profiles on MathSciNet. In total we collected citation records for 2807 math professors at 131 different institutions. We then collected the total citations, the year of earliest indexed publication, and the field of the most cited publication for each mathematician. A second pass through the data set occurred between January 30 - February 2, 2020, and discovered errors were corrected. This data set is as complete as possible.

There were initially 6565 fields that were most cited as classified by MathSciNet, which we reduced to 2020 fields, using the mapping in the appendix. This mapping was constructed using general expertise on the way each field worked, and is recorded in the appendix.

2. Exploratory Data Analysis

2.1. Distribution

Citations and Citations/Year1.3 appear exponentially distributed after transformation by a square root.

[Uncaptioned image]

Figure 1: Distribution of citations and an exponential qq-plot of square root citations.

[Uncaptioned image]

Figure 2: Distribution of citations per year1.3 and an exponential qq-plot of square root citations per year1.3.

2.2. Citations/Year1.3 vs. Age

As noted in earlier work [2], citations and year are positively correlated, but citations per year1.3 and year are not correlated111The exponent 1.31.3 was determined via a statistical analysis. We repeat this analysis to check for robustness and find that when linearly regressed, citations/year1.3 and year have a slope of 0.03380.0338 with a 95% confidence interval of [0.00899,0.0766][-0.00899,0.0766]. The pp-value is 0.1220.122, so we fail to reject the null hypothesis that the slope is zero, and R2=0.03R^{2}=0.03. We conclude that citations/year1.3 and year are not correlated.

[Uncaptioned image]

Figure 3: Scatter plot of citations/year1.3 and Years elapsed since first publication. The red line indicates the regressed line with equation 0.0338Year+8.40=Citations/Year1.30.0338Year+8.40=Citations/Year^{1.3}. The pp-value for slope is 0.1220.122 and the R2=0.03R^{2}=0.03. We fail to reject the null hypothesis that the slope is zero and conclude they are not correlated.

2.3. Factored Linear Models

Before proceeding with the analysis, we should assess the importance of the explanatory variables when looking at differences in citations between mathematicians. We do this by constructing nested linear models with all three combinations of Year and Field, and determine the best model with Akaike Information Criterion, AIC [1]. Per standard interpretation the lower the AIC, the better the model. Let C=C=Citations, a=a= Age, and f=f= Field.

Model AIC score
log(C)=β0a+β1f+ϵ\log(C)=\beta_{0}a+\beta_{1}f+\epsilon 9122.1249122.124
log(C)=β0f+ϵ\log(C)=\beta_{0}f+\epsilon 9347.3189347.318
log(C)=β0a+ϵ\log(C)=\beta_{0}a+\epsilon 9514.6449514.644

Figure 4: Table of AIC scores for tested linear models. As the AIC score is lowest, the model consisting of both Age and Field is best, where Age impacts citations positively. For more detailed information on the model, refer to the ANOVA tables in the RMarkdown on Github. It is clear that certain fields contribute negatively to overall citations and other fields contribute positively.

While it is not appropriate to pick a model for the sole reason that it minimizes AIC, it makes sense to consider both age and field.

2.4. Fields

Different fields in mathematics have different citation practices. Some fields like Partial Differential Equations have more mathematicians, whereas some fields like Number Theory have fewer mathematicians. Some results from fields like topology are widely applicable across disciplines, whereas more obscure results are not. We quantify the bibliographic differences between fields. Note that the major fields below are larger categories containing potentially multiple MathSciNet tags, and the mappings are recorded in the appendix.

Major Field Mean Citations S.D. Citations Mean Cit/Year1.3 Count
PDE 1472.07 2182.45 14.58 372
Computer Science 1260.44 2223.06 14.08 225
Probability 1165.92 1401.07 12.06 137
Harmonic Analysis 1120.12 1336.62 10.51 200
Combinatorics 1023.24 1673.53 10.08 116
Algebra 934.42 1310.59 9.12 220
Algebraic Geometry 846.62 1308.80 9.51 169
Geometry 890.68 1486.72 8.87 311
Number Theory 742.66 920.31 7.38 159
Dynamics 560.44 555.17 7.33 68
Mathematical Physics 643.01 716.41 7.25 96
Analysis 977.18 1951.95 7.15 45
Applied Mathematics 646.60 976.98 6.87 299
Group Theory 686.38 1151.64 6.74 81
Logic 634.00 690.14 6.32 55
Complex Analysis 612.86 725.15 6.17 115
Lie Groups 512.02 590.59 4.78 43
Statistics 220.73 331.15 3.10 83
History 74.0 104.65 0.677 2
Other 5 6.61 0.074 11

Figure 5: Mean citations and citations per year1.3 including counts, split by field, from top to bottom ranked by mean citations per year1.3 to account for age.

We ran a permutation test between each field to verify the observed partial order. We report the inconclusive differences (pp-value greater than 0.050.05) between fields when comparing citations per year1.3 in the appendix.

2.5. Ranking of Departments

The above figures shows that comparing mathematicians in two different fields is akin to comparing apples and oranges. The cleanest way to standardize this is to compute an interfield z-score of the citations per year1.3, and hence associating a “rank” to each mathematician. Then we computed the mean of the interfield zz-scores of each full professor at the respective institution. We report here the top 20 schools and record the remaining schools in the appendix222Where by ”Schools” we mean ”Mathematics departments” - for Universities with separate Pure and Applied math departments, the ranking will be different if the departments were to be combined.

  1. (1)

    Princeton University

  2. (2)

    Harvard University

  3. (3)

    Stanford University

  4. (4)

    University of Chicago

  5. (5)

    Columbia University in the City of New York

  6. (6)

    Massachussetts Institute of Technology

  7. (7)

    University of California, Los Angeles

  8. (8)

    University of Miami

  9. (9)

    Yale University

  10. (10)

    Brown University

  11. (11)

    University of California, Berkeley

  12. (12)

    New York University

  13. (13)

    University of Oregon

  14. (14)

    California Institute of Technology

  15. (15)

    Duke University

  16. (16)

    Stony Brook University

  17. (17)

    Rutgers University-New Brunswick

  18. (18)

    University of Virginia

  19. (19)

    Texas A&M University

  20. (20)

    Northwestern University

3. Conclusion

The rankings based on our normalized zz-score (call it the PR score) correspond reasonably well with the “folk” rankings of mathematicians. While we do not want to flatter or insult individuals by giving their scores here, we do give a ranking of departments, and we see that it, again, corresponds well with the “folk” rankings. If they do not, we encourage the reader to look at the faculty pages of the departments in question. It seems, therefore, that there is, indeed, a fully quantitative way to produce meaningful rankings which work at least in a statistical sense - they fail for polymaths, and they also are less successful for mathematicians the bulk of whose work is not indexed by MathSciNet - in particular those who do interdisciplinary work.

References

  • [1] Christopher M Bishop. Machine learning and pattern recognition. 2006.
  • [2] Joshua Paik and Igor Rivin. Data analysis of the responses to professor abigail thompson’s statement on mandatory diversity statements, 2020.

4. Appendix

4.1. Code and Data

Available at https://github.com/joshp112358/Differences .

4.2. Classifications

Major Field Sub Fields
Algebra Algebraic Topology; Associative
Rings and Algebra; Category theory, Homological algebra ;
Commutative rings and algebras; Field theory;
General algebraic systems; K-theory; Linear
and Multilinear Algebra, matrix theory; Associative
rings and algebras; Order, lattices, ordered algebraic
structures.
Algebraic Geometry Algebraic Geometry
Analysis Difference and functional equations;
Integral equations; Integral transforms, operational calculus;
Ordinary differential equations; Real functions;
Special functions.
Applied Mathematics Approximations and expansions; Biology
other natural sciences; Calculus of variations and optimal
control, optimization; Fluid mechanics,
Game theory, economics, social and behavioral sciences;
Geophysics, Mechanics of deformable sciences;
Mechanics of solids, Operations research, mathematical
programming; Systems theory, control.
Combinatorics Combinatorics
Complex Analysis Functions of a complex variable; Potential theory; Several
complex variables and analytic spaces
Computer Science Computer Science; Numerical Analysis;
Information and communication, circuits.
Dynamics Dynamical Systems and Ergodic Theory
Geometry Convex and discrete geometry; Differential Geometry;
General topology; Geometry;
Manifolds and cell complexes;
Group theory Group theory and generalizations.
Harmonic analysis Abstract harmonic analysis; Fourier analysis;
Functional analysis; Global analysis, analysis on manifolds;
Measure and integration, Operator theory.
History History and biography.
Lie Groups Topological Groups, Lie Groups.
Logic Logic and foundations; Mathematical logic and foundations;
Set theory
Major Field Sub Fields
Mathematical Physics Classical thermodynamics, heat transfer;
Mechanics of particles and systems; Optics, electromagnetic
theory; Quantum theory; Relativity and gravitational theory;
Statistical mechanics, structure of matter.
Number Theory Number Theory
Other Other
PDEs Partial Differential Equations; Global Analysis,
Analysis on manifolds
Probability Probability theory and stochastic processes
Statistics Statistics

4.3. Ranking of Departments

  1. (1)

    Princeton University

  2. (2)

    Harvard University

  3. (3)

    Stanford University

  4. (4)

    University of Chicago

  5. (5)

    Columbia University in the City of New York

  6. (6)

    Massachussetts Institute of Technology

  7. (7)

    University of California, Los Angeles

  8. (8)

    University of Miami

  9. (9)

    Yale University

  10. (10)

    Brown University

  11. (11)

    University of California, Berkeley

  12. (12)

    New York University

  13. (13)

    University of Oregon

  14. (14)

    California Institute of Technology

  15. (15)

    Duke University

  16. (16)

    Stony Brook University

  17. (17)

    Rutgers University-New Brunswick

  18. (18)

    University of Virginia

  19. (19)

    Texas A&M University

  20. (20)

    Northwestern University

  21. (21)

    University of Michigan

  22. (22)

    Rice University

  23. (23)

    The University of Texas at Austin

  24. (24)

    Carnegie Mellon University

  25. (25)

    University of Illinois at Chicago

  26. (26)

    University of California, Irvine

  27. (27)

    University of Pittsburgh

  28. (28)

    Georgia Institute of Technology

  29. (29)

    University of Minnesota

  30. (30)

    Vanderbilt University

  31. (31)

    Indiana University Bloomington

  32. (32)

    SUNY at Albany

  33. (33)

    University of California, San Diego

  34. (34)

    University of North Texas

  35. (35)

    University of Washington

  36. (36)

    University of Connecticut

  37. (37)

    Arizona State University

  38. (38)

    Pennsylvania State University

  39. (39)

    University of Southern California

  40. (40)

    Purdue University

  41. (41)

    University of Illinois at Urbana-Champaign

  42. (42)

    Cornell University

  43. (43)

    University of Maryland - College Park

  44. (44)

    University of Utah

  45. (45)

    North Carolina State University

  46. (46)

    Johns Hopkins University

  47. (47)

    University of California, Riverside

  48. (48)

    University of California, Santa Cruz

  49. (49)

    Washington University in St. Louis

  50. (50)

    Wayne State University

  51. (51)

    University of Pennsylvania

  52. (52)

    Brandeis University

  53. (53)

    Colorado State University

  54. (54)

    University of Notre Dame

  55. (55)

    University of California, Santa Barbara

  56. (56)

    University of North Carolina at Chapel Hill

  57. (57)

    University of Houston

  58. (58)

    University of Iowa

  59. (59)

    The Ohio State University

  60. (60)

    University of South Florida

  61. (61)

    Michigan State University

  62. (62)

    University of California, Davis

  63. (63)

    Virginia Polytechnic Institute and State University

  64. (64)

    University of Missouri

  65. (65)

    University of Wisconsin - Madison

  66. (66)

    University of Massachusetts Amherst

  67. (67)

    University of South Carolina

  68. (68)

    Emory University

  69. (69)

    University of Central Florida

  70. (70)

    University of Kentucky

  71. (71)

    University of Florida

  72. (72)

    University of Delaware

  73. (73)

    Louisiana State University

  74. (74)

    Syracuse University

  75. (75)

    Georgia State University

  76. (76)

    University of Colorado Denver

  77. (77)

    Boston University

  78. (78)

    Tulane University of Louisiana

  79. (79)

    Clemson University

  80. (80)

    University of Kansas

  81. (81)

    University of Southern Mississippi

  82. (82)

    Boston College

  83. (83)

    Mississippi State University

  84. (84)

    University of Rochester

  85. (85)

    CUNY Graduate School and University Center

  86. (86)

    The University of Tennessee

  87. (87)

    George Washington University

  88. (88)

    Georgetown University

  89. (89)

    Florida State Universty

  90. (90)

    Iowa State University

  91. (91)

    University at Buffalo

  92. (92)

    Northeastern University

  93. (93)

    Tufts University

  94. (94)

    University of Nebraska-Lincoln

  95. (95)

    University of Georgia

  96. (96)

    University of New Hampshire

  97. (97)

    Virgina Commonwealth

  98. (98)

    University of Cincinatti

  99. (99)

    Dartmouth College

  100. (100)

    Rennselaer Polytechnic Institute

  101. (101)

    University of Nevada, Reno

  102. (102)

    West Virginia University

  103. (103)

    Auburn University

  104. (104)

    The University of Texas at Arlington

  105. (105)

    Texas Tech University

  106. (106)

    University of Arizona

  107. (107)

    Binghamton University

  108. (108)

    University of New Mexico

  109. (109)

    The University of Alabama

  110. (110)

    The University of Texas at Dallas

  111. (111)

    George Mason University

  112. (112)

    Florida Institute University

  113. (113)

    University of Oklahoma

  114. (114)

    University of Colorado Boulder

  115. (115)

    University of Hawaii at Manoa

  116. (116)

    Case Western Reserve University

  117. (117)

    University of Alabama at Birmingham

  118. (118)

    Oklahoma State University

  119. (119)

    Kansas State University

  120. (120)

    Temple University

  121. (121)

    Oregon State University

  122. (122)

    Drexel University

  123. (123)

    University of Louisville

  124. (124)

    University of Nevada, Las Vegas

  125. (125)

    University of Wisconsin - Milwaukee

  126. (126)

    Washington State University

  127. (127)

    New Jersey Institute of Technology

  128. (128)

    The University of Texas at El Paso

  129. (129)

    University of Mississippi

  130. (130)

    University of Arkansas

  131. (131)

    Montana State University

4.4. Inconclusive Permutation Tests between Fields

We report the results of a one sided permutation test, when comparing cit/year1.3 which failed to be significant at the 0.050.05 level. We record the hypothesis on the left and the pp-value to the right.

PDE \geq Computer Science — pp-value: 0.397

PDE \geq Probability — pp-value: 0.0768

Computer Science \geq Probability — pp-value: 0.156

Probability \geq Harmonic Analysis — pp-value: 0.113

Probability \geq Combinatorics — pp-value: 0.1049

Harmonic Analysis \geq Combinatorics — pp-value: 0.3824

Harmonic Analysis \geq Algebra — pp-value: 0.0961

Harmonic Analysis \geq Algebraic Geometry — pp-value: 0.1807

Combinatorics \geq Algebra — pp-value: 0.2181

Combinatorics \geq Algebraic Geometry — pp-value: 0.3265

Combinatorics \geq Geometry — pp-value: 0.1544

Combinatorics \geq Analysis — pp-value: 0.0719

Algebra \geq Algebraic Geometry — pp-value: 0.6461

Algebra \geq Geometry — pp-value: 0.3989

Algebra \geq Dynamics — pp-value: 0.0813

Algebra \geq Mathematical Physics — pp-value: 0.0546

Algebra \geq Analysis — pp-value: 0.1232

Algebraic Geometry \geq Geometry — pp-value: 0.2533

Algebraic Geometry \geq Analysis — pp-value: 0.0782

Geometry \geq Number Theory — pp-value: 0.0534

Geometry \geq Dynamics — pp-value: 0.1076

Geometry \geq Mathematical Physics — pp-value: 0.0704

Geometry \geq Analysis — pp-value: 0.1497

Number Theory \geq Dynamics — pp-value: 0.4906

Number Theory \geq Mathematical Physics — pp-value: 0.4514

Number Theory \geq Analysis — pp-value: 0.4417

Number Theory \geq Applied Mathematics — pp-value: 0.2717

Number Theory \geq Group Theory — pp-value: 0.2637

Number Theory \geq Logic — pp-value: 0.1557

Number Theory \geq Complex Analysis — pp-value: 0.0717

Dynamics \geq Mathematical Physics — pp-value: 0.4614

Dynamics \geq Analysis — pp-value: 0.4609

Dynamics \geq Applied Mathematics — pp-value: 0.3377

Dynamics \geq Group Theory — pp-value: 0.2999

Dynamics \geq Logic — pp-value: 0.1738

Dynamics \geq Complex Analysis — pp-value: 0.1272

Mathematical Physics \geq Analysis — pp-value: 0.4916

Mathematical Physics \geq Applied Mathematics — pp-value: 0.3506

Mathematical Physics \geq Group Theory — pp-value: 0.337

Mathematical Physics \geq Logic — pp-value: 0.23

Mathematical Physics \geq Complex Analysis — pp-value: 0.1423

Mathematical Physics \geq History — pp-value: 0.0525

Analysis \geq Applied Mathematics — pp-value: 0.4035

Analysis \geq Group Theory — pp-value: 0.3972

Analysis \geq Logic — pp-value: 0.3154

Analysis \geq Complex Analysis — pp-value: 0.2375

Analysis \geq Lie Groups — pp-value: 0.1041

Analysis \geq History — pp-value: 0.0881

Applied Mathematics \geq Group Theory — pp-value: 0.4655

Applied Mathematics \geq Logic — pp-value: 0.3609

Applied Mathematics \geq Complex Analysis — pp-value: 0.2362

Applied Mathematics \geq Lie Groups — pp-value: 0.0616

Applied Mathematics \geq History — pp-value: 0.0534

Group Theory \geq Logic — pp-value: 0.3728

Group Theory \geq Complex Analysis — pp-value: 0.2834

Group Theory \geq Lie Groups — pp-value: 0.0513

Logic \geq Complex Analysis — pp-value: 0.4276

Logic \geq Lie Groups — pp-value: 0.0613

Complex Analysis \geq Lie Groups — pp-value: 0.0871

Lie Groups \geq History — pp-value: 0.0531

Statistics \geq History — pp-value: 0.2517

History \geq Other — pp-value: 0.1533