asked    Simon     2018-10-22       python       71 view        1 Answer

[SOLVED] Negative Binomial Loss in Neural Network using Tensorflow / Keras

I'm working with a highly skewed count dataset that's typically fit with a negative binomial. I want to use the negative binomial as a loss functions in Keras or Tensorflow on a feed forward neural network.

To my knowledge, after looking through available loss functions, such a function doesn't exist for keras or tensorflow (although I'm hoping I'm wrong and I just missed something).

I've looked around and I've seen posts suggest just swapping out the loss to include the negative binomial, but it appears there's more to creating a custom loss in tensorflow than just swapping it out with tf.contrib.distributions.NegativeBinomial

There are several posts that talk about creating custom loss functions for Tensorflow / keras, e.g.: After looking at this, I'm struggling to figure out how to code this.

I'm wondering if 1) anyone has already created a loss function using negative binomial and is willing to share how it was coded or 2) tips on how to make this work.

I'm working exclusively in python on this project.

Appreciate all the help in advance.

  1 Answer  

        answered    Veronica     2018-10-22      

I don't know about negative binomial loss, but I do know how to implement a custom loss function

def custom_loss(y_true, y_pred):
    ...
    loss = ...
    return loss

Treat y_true and y_pred as tensors. (this means instead of using functions such as np.pow(y_true,2) you need to use functions from tensorflow or keras.backend)

pass your custom loss function as compile argument

model.compile(loss=custom_loss, optimizer='your favorite optimizer')




Your Answer





 2018-10-22         Phoebe

Opening .bcp files in R

I have been trying to convert UK charity commission data which is in .bcp file format into .csv file format which could then be read into R. The data I am referring to is available here: http://data.charitycommission.gov.uk/. What I am trying to do is turn these .bcp files into useable dataframes that I can clean and run analyses on in R.There are suggestions on how to do this through python on this github page https://github.com/ncvo/charity-commission-extract but unfortunately I haven't been able to get these options to work. I am wondering if there is any syntax or packa...
 r                     1 answers                     53 view
 2018-10-22         Michaelia

How to export schema of a database in AppMaker

I used the following command from shell to export the schema of the database:mysqldump -u username -p -no-data database_name> gs://test.sqlI got the following error:mysqldump: unknown option '-o'. Then instead I used this:mysqldump -u username -p -d database_name, ( -d instead of -no-data flag)I got this error:Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2 "No such file or directory")When trying to connect.Any idea how to export only table definitions in the schema and not all the database. You're missing the hostname.mysqldump -h ...
 appmaker                     1 answers                     50 view
 2018-10-22         Duke

How to understand node-soap "describe" function?

I'm attempting to use the node SOAP npm module(https://github.com/vpulim/node-soap) service. var soap = require('soap');var soapWSDL = "https://webservice.s7.exacttarget.com/etframework.wsdl";soap.createClient(soapWSDL, function (err, client) { if (err) { return callback(err, null); } client.setSecurity(new soap.WSSecurity(self.username, self.password)); console.log("describe", client.describe()); console.log("retrieve", client.describe().PartnerAPI.Soap.Retrieve);});The first log shows the available methods... But i'm trying to understand the exact for...
 node.js                     2 answers                     51 view