Sites built with cmDRUPAL

    • pcam

How to update nodes with feeds from multiple sources (and setting unique fields)

At RETN they ran into a problem where they have a lot of legacy VOD on Amazon from when they were on a Synergy system. They also have thousands of shows in their Drupal site that have a field called "streaming_filename" that holds the filename to these legacy shows.

Then they decided to move to Telvue Hypercaster and also Cloudcast, but they needed a way to bulk update all the shows with the new Cloudcast VOD.

What they decided to do was to keep the filename the same in Telvue so that the filename would ultimately be the same in Cloudcast.

Meaning, in their old system, they have a show called "Show #1" that had an MP4 file called "RETN-ATF-001.mp4". In their drupal field field_streaming_filename is the value "RETN-ATF-001". So they put that same filename into Telvue as one would expect, and so it comes through to Cloudcast also as "RETN-ATF-001.mp4". However, their old system was not using media_cloudcast module so in order to get that up and running we needed to then get the cloudcast url into the applicable node.

If you've used feeds and feed importers, you may see the issue here. We needed the feed importer to match the filename from the Cloudcast MRSS feed to the field_streaming_filename field in Drupal. If it finds a match, it should update that show's media field with the Cloudcast vod. If it doesn't find a match, it should add a new show.

Unfortunately, by default feeds does not hook in to unique identifiers on any field. So it was just creating duplicate shows (even if you set a GUID, that is just a mechanism for feeds to know it previously ran an import).

I documented how I resolved this in the feed's issue queue on a very long thread:

But considering the length of that thread, here is a point-by-point way to do something similar on your system if there is any field you are trying to make feeds recognize as unique in order to update nodes properly. The best thing to do would be to subscribe to both issue [#661606] and issue [#1705386] so you can stay updated on if these get rolled in to either module mentioned below.

  • Install the field validation module 7-x.2-3, then download this zip file and copy the enclosed file into the field_validation module folder (yes, overwrite the existing file, obviously this is not ideal solution as patching but that's what seems to work now)
  • Then you will need to patch your feeds module. I am using the generic entity processor branch of feeds which can be found on feed's version control page, but it should also work with the latest dev release. The patch file you need is in comment #111 of [#661606] (feeds-unique-target-661606-111.patch)
  • Once you do the above and enable field validation module, go into the content type > Manage Fields that you want to set a unique field on. Now you will have a 'validate' link next to every field. click that for the field you want to change. Then apply the unique validator. Save.
  • Now create your feed importer. In RETN's case we also want to pull cloudcast episode tag from the MRSS feed, so for their feed importer we are using the "MediaRSS Cloudcast parser" and then for the processor we have tested using both just regular node processor and entity processor and both work to hook into shows.
  • In the mapping, when we add a target for field_streaming_filename, we now get the "unique" checkbox. Note that as with all of these, by default it will be off. You not only have to click "update" and check the box, but if you don't then save the overall screen it will not retain your changes.

Now when this import runs, it updates the correct show and now the cloudcast media displays:

Note that you could use this approach to also have multiple feeds updating the same nodes, so it gives you a lot more control.

Attachment Size
cloudcast1.png 22.69 KB
srb.png 342.77 KB