Skip to content

Sum of date and interval on deferred columns cannot infer types #11797

@frapa

Description

@frapa

What happened?

When I try to sum a date with an interval, if the interval is based on a deferred expression, ibis throws a InputTypeError: Unable to infer datatype of value error. I think this is because it tries to evaluate the type of a deferred too early, before it is resolved.

Minimum example to reproduce:

import ibis

tab = ibis.memtable({"days": [0, 10, 100, 365]})

# This works
tab.select(ibis.date(1970, 1, 1) + tab.days.as_interval("D")).to_pandas()
#   DateAdd(DateFromYMD(1970, 1, 1), IntervalFromInteger(days, DAY))
# 0                                         1970-01-01              
# 1                                         1970-01-11              
# 2                                         1970-04-11              
# 3                                         1971-01-01              

# This fails
tab.select(ibis.date(1970, 1, 1) + ibis.deferred["days"].as_interval("D")).to_pandas()
# ibis.common.exceptions.InputTypeError: Unable to infer datatype of value 
# _['days'].as_interval('D') with type <class 'ibis.common.deferred.Deferred'>

What version of ibis are you using?

11.0.0

What backend(s) are you using, if any?

DuckDB

Relevant log output

Traceback (most recent call last):
  File "/home/francesco/Projects/spark-ibis/.venv/lib/python3.12/site-packages/ibis/expr/operations/core.py", line 70, in __coerce__
    dtype = dt.DataType.from_typehint(T)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/francesco/Projects/spark-ibis/.venv/lib/python3.12/site-packages/ibis/expr/datatypes/core.py", line 309, in from_typehint
    raise TypeError(f"Value {typ!r} is not a valid datatype")
TypeError: Value typing.Annotated[ibis.expr.datatypes.core.Interval, Attrs(fields={'unit': CoercedTo(func=<bound method Unit.__coerce__ of <enum 'DateUnit'>>, type=<enum 'DateUnit'>)})] is not a valid datatype
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/francesco/.local/share/JetBrains/Toolbox/apps/pycharm-professional/plugins/python-ce/helpers/pydev/pydevconsole.py", line 364, in runcode
    coro = func()
           ^^^^^^
  File "<input>", line 1, in <module>
  File "/home/francesco/Projects/spark-ibis/.venv/lib/python3.12/site-packages/ibis/expr/types/temporal.py", line 623, in __add__
    return _binop(ops.DateAdd, self, other)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/francesco/Projects/spark-ibis/.venv/lib/python3.12/site-packages/ibis/expr/types/core.py", line 977, in _binop
    node = op_class(left, right)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/francesco/Projects/spark-ibis/.venv/lib/python3.12/site-packages/ibis/common/bases.py", line 72, in __call__
    return cls.__create__(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/francesco/Projects/spark-ibis/.venv/lib/python3.12/site-packages/ibis/common/grounds.py", line 119, in __create__
    kwargs = cls.__signature__.validate(cls, args, kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/francesco/Projects/spark-ibis/.venv/lib/python3.12/site-packages/ibis/common/annotations.py", line 494, in validate
    result = pattern.match(value, this)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/francesco/Projects/spark-ibis/.venv/lib/python3.12/site-packages/ibis/common/patterns.py", line 876, in match
    value = self.origin.__coerce__(value, **self.params)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/francesco/Projects/spark-ibis/.venv/lib/python3.12/site-packages/ibis/expr/operations/core.py", line 72, in __coerce__
    dtype = dt.infer(value)
            ^^^^^^^^^^^^^^^
  File "/home/francesco/Projects/spark-ibis/.venv/lib/python3.12/site-packages/ibis/common/dispatch.py", line 115, in call
    return impl(arg, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/francesco/Projects/spark-ibis/.venv/lib/python3.12/site-packages/ibis/expr/datatypes/value.py", line 35, in infer
    raise InputTypeError(
ibis.common.exceptions.InputTypeError: Unable to infer datatype of value _['days'].as_interval('D') with type <class 'ibis.common.deferred.Deferred'>

Code of Conduct

  • I agree to follow this project's Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugIncorrect behavior inside of ibis

    Type

    No type

    Projects

    Status

    backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions