### Pandas version checks - [X] I have checked that this issue has not already been reported. - [X] I have confirmed this bug exists on the [latest version](https://pandas.pydata.org/docs/whatsnew/index.html) of pandas. - [ ] I have confirmed this bug exists on the [main branch](https://pandas.pydata.org/docs/dev/getting_started/install.html#installing-the-development-version-of-pandas) of pandas. ### Reproducible Example The following started failing [one week ago](https://github.com/mwouts/itables/actions/runs/6036501048), with `pandas==2.1.0`: ```python def test_datetime_diff(date_min="1677-09-23", date_max="2262-04-10"): return pd.to_datetime([date_min, date_max]).diff() ``` ### Issue Description ``` if to_raise: > raise OverflowError("Overflow in int64 addition") E OverflowError: Overflow in int64 addition ``` ### Expected Behavior Taking a difference on datetime objects should not raise any exception and return a timedelta. I have also tried a less extreme date difference, which yields a different error: ``` def test_datetime_diff_non_extreme_dates(date_min="1997-09-23", date_max="2001-04-10"): return pd.to_datetime([date_min, date_max]).diff() ``` ``` # GH#29794 enforcing deprecation introduced in GH#23539 > raise TypeError(f"dtype {data.dtype} cannot be converted to datetime64[ns]") E TypeError: dtype timedelta64[ns] cannot be converted to datetime64[ns] ../../miniconda3/envs/itables/lib/python3.11/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils. warnings.warn("Setuptools is replacing distutils.") INSTALLED VERSIONS ------------------ commit : ba1cccd19da778f0c3a7d6a885685da16a072870 python : 3.11.4.final.0 python-bits : 64 OS : Linux OS-release : 6.2.0-31-generic Version : #31~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Wed Aug 16 13:45:26 UTC 2 machine : x86_64 processor : x86_64 byteorder : little LC_ALL : None LANG : en_US.UTF-8 LOCALE : en_US.UTF-8 pandas : 2.1.0 numpy : 1.25.2 pytz : 2022.7 dateutil : 2.8.2 setuptools : 68.0.0 pip : 23.2.1 Cython : None pytest : 7.4.0 hypothesis : None sphinx : 5.0.2 blosc : None feather : None xlsxwriter : None lxml.etree : 4.9.2 html5lib : None pymysql : None psycopg2 : None jinja2 : 3.1.2 IPython : 8.12.2 pandas_datareader : None bs4 : 4.12.2 bottleneck : 1.3.5 dataframe-api-compat: None fastparquet : None fsspec : None gcsfs : None matplotlib : None numba : None numexpr : 2.8.4 odfpy : None openpyxl : None pandas_gbq : None pyarrow : 11.0.0 pyreadstat : None pyxlsb : None s3fs : None scipy : None sqlalchemy : 2.0.20 tables : None tabulate : 0.9.0 xarray : None xlrd : None zstandard : None tzdata : 2023.3 qtpy : 2.2.0 pyqt5 : None >>> ``` </details>