How to use Keras FeatureSpace to ingest a dataframe that has an embedding column?

Keras
1

Hi there! I’m following the example that reads in a pandas dataframe using FeatureSpace in Structured data classification with FeatureSpace - the example works out of the box. I am now trying to modify the example:

One of my dataframe columns has text that I am sending to OpenAI’s embeddings endpoint and it returns a length 1536 float list. I want to ingest this float list into Keras. One idea is to create 1536 new float columns in the dataframe and ingest each of these columns into keras. Is that the proper way to do it? Or is there a way to ingest a column that is a list of floats? Thank you!

3

Hi,

Does your return 1536 float list is same length as your dataframe size ?
If yes then this 1536 float list you can add it into dataframe new column. And add this column name into keras.utils.FeatureSpace
Here is the example:

feature_space = FeatureSpace(
    features={
        #new float column= new_data column name
        "new_data": FeatureSpace.float_normalized()
        # Categorical features encoded as integers
        "sex": FeatureSpace.integer_categorical(num_oov_indices=0),
        "age": FeatureSpace.float_discretized(num_bins=30),
        # Numerical features to normalize
        "trestbps": FeatureSpace.float_normalized(),
        "oldpeak": FeatureSpace.float_normalized(),
        "slope": FeatureSpace.float_normalized(),
    },
    # Specify feature cross with a custom crossing dim.
    crosses=[
        FeatureSpace.cross(feature_names=("sex", "age"), crossing_dim=64),
        FeatureSpace.cross(
            feature_names=("thal", "ca"),
            crossing_dim=16,
        ),
    ],
    output_mode="concat",
)