Kettle Palo Plugin

This site uses cookies. By continuing to browse this site, you are agreeing to our Cookie Policy.

  • Kettle Palo Plugin

    Hi!

    I have downloaded Pentaho Data Integration (prev Kettle) version 3.03.
    I have also installed the Palo Plugin that is found in sourceforge.net/projects/palokettleplug/

    I had to download the Jpalo api version 1.5, move jpalo.jar to kettle/libext folder.

    I have tested it and trying it out.
    So far the dimension output to Palo works but slow.

    Anyone else trying it out?
    Care to share your experiences?
  • RE: Kettle Palo Plugin

    hi sentosa,

    the kettle plugin works fine with one exception. when i'm importing dimensions in palo with kettle i always get text formatted dimensions, with those i doen't see any data in the cube. (happens always when the fields are defined as string, with integer formatted fields it works properly)

    may you can give me an advise how to format or transform the database input files???

    greatings
    markus
  • RE: Kettle Palo Plugin

    Hi sentosa,

    Just like you I'm trying to figure out how to do data import
    with
    - the Pentaho Data Integration (PDI) - i'm using the new 3.1.0-RC1 link
    - with Palo 2.5 (not 2.0!) link
    - Palo Java api 2.0 (designed for Palo 2.0) link
    - and the latest PaloKettlePlugin 1.0 (designed for Palo 2.0) from Jan 28 2008 link

    I do not know which versions you have working, but I'd appreciate some assistance.
    my Demo database is up and running
    jpalo.jar and PaloKettle.jar are in libext directory of PDI
    as the documentation describes.

    I'm trying to set up the connection in PDI to the Demo database on localhost as described in the PaloKettePlugin documentation, but get an error message when testing.

    (please see images attached)

    I have also tried to open the connection from the PaloDimension Output block... but get
    an error "Dimension does not exist"
    when pressin the "Get Current Levels" button for the block

    I'm afraid that this is due to the incompatibility between the PaloKettlePlugin (dated January 2008 and the new 2.5 version of Palo ... Or the new version of Kettle (=PDI)

    All Help appreciated - if somebody has figured out how to get Palo 2.5 to work with new 3.0+ version of Kettle.
    Does any body know anything about planned future updates to the Kettle plugin???

    Thanks
    G.

    p.s. similar issues have been raised in Pantaho forum
    link
    The mentioned remedy is to use the OLDER version of Palo Java API 1.5
    I don't quite understand why.
    Images
    • DatabaseConnection.gif

      42.23 kB, 701×610, viewed 7,025 times
    • Test error.gif

      79.27 kB, 620×766, viewed 3,625 times
    • DimensionOutput.gif

      49.6 kB, 905×673, viewed 3,618 times

    The post was edited 1 time, last by gblasko ().

  • The PaloKettlePlugin 1.0 DOES work with
    - Pentaho Data Integration (PDI) 3.1.0-RC1
    - Palo 2.5
    - Palo Java API Build 2.0.0.214

    BUT it has the following limitations
    - testing the database connection does not work (it always shows an error message), but actual transformations works OK, if the connection is defined correct.
    - all elements are hardcoded to be created as strings which seldom are useful, because data in cubes will not be consolidated in dimension hierarchies.

    I have made my own version 1.1 of the Plugin which has the following additions:
    - option to choose the Default Element Type (Numeric or String) for new elements being loaded into a dimension.
    - option to clear (delete) all elements from dimension before loading new elements.
    - option to clear (delete) all values from cube before loading new values.

    The Test button in the database connection for a Palo database has not been fixed, as I could not figure out how to do this.

    It is also not compatible with transformations that contains "Palo Dimension Output" or "Palo Cells Output" steps created with version 1.0 of the plugin.

    The attached zip file contains both a ready-to-be-used compiled jar and the source code.


    I have also tried to open the connection from the PaloDimension Output block... but get an error "Dimension does not exist" when pressin the "Get Current Levels" button for the block


    The Palo Dimension Output step is used for inserting elements into a dimension in your Palo database. I see in your last screenshot that you have selected a Connection "Palo Demo", but you have not selected a dimension in the drop down list just above the "Get Current Levels" button. That is the reason you get the error "Dimension" does not exist".

    Kind regards,
    Bent
    Files
  • Is that last version ?

    Hi all,

    I've copied new version (11) you posted here and it seems that it works.
    The Sourceforge is not updated to this version or is it the same ?
    Many thanks
    There is still the connection test error but it's not blocking .
    Is that correct ?
    The errror:

    Errore di connessione al database [classicmodels] : java.lang.InstantiationException: plugin.palo.core.PaloHelper
    java.lang.InstantiationException: plugin.palo.core.PaloHelper
    at java.lang.Class.newInstance0(Unknown Source)
    at java.lang.Class.newInstance(Unknown Source)
    at org.pentaho.di.core.database.DatabaseMeta.getDatabaseFactory(DatabaseMeta.java:2495)
    at org.pentaho.di.core.database.DatabaseMeta.testConnection(DatabaseMeta.java:2477)
    at org.pentaho.di.ui.core.database.dialog.DatabaseDialog.test(DatabaseDialog.java:1919)
    at org.pentaho.di.ui.core.database.dialog.DatabaseDialog.test(DatabaseDialog.java:1901)
    at org.pentaho.di.ui.core.database.dialog.DatabaseDialog$4.handleEvent(DatabaseDialog.java:293)
    at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
    at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
    at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
    at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
    at org.pentaho.di.ui.core.database.dialog.DatabaseDialog.open(DatabaseDialog.java:368)
    at org.pentaho.di.ui.spoon.delegates.SpoonDBDelegate.newConnection(SpoonDBDelegate.java:505)
    at org.pentaho.di.ui.spoon.Spoon.doubleClickedInTree(Spoon.java:2284)
    at org.pentaho.di.ui.spoon.Spoon.access$500(Spoon.java:240)
    at org.pentaho.di.ui.spoon.Spoon$8.widgetDefaultSelected(Spoon.java:1343)
    at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
    at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
    at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
    at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
    at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
    at org.pentaho.di.ui.spoon.Spoon.readAndDispatch(Spoon.java:873)
    at org.pentaho.di.ui.spoon.Spoon.start(Spoon.java:5639)
    at org.pentaho.di.ui.spoon.Spoon.run(Spoon.java:5735)
    at org.pentaho.di.ui.spoon.Spoon.main(Spoon.java:371)
  • I've copied new version (11) you posted here and it seems that it works. The Sourceforge is not updated to this version or is it the same ?

    I did not make the original version available on sourceforge. I have sent a mail to the people behind the original plugin with my updated version, but I have not received a response yet. So it is still the original version 1.0 that is available on sourceforge.

    There is still the connection test error but it's not blocking . Is that correct ?


    As I mentioned in my previous post I could not figure out how to fix the Test button, so this does still not work. But transformations will work OK, as long as the connection information are correct.
  • RE: Kettle Palo Plugin

    Hi folks, good news:
    We have added some features, like using attribute dimensions and cubes. In addition we have fixed some bugs, like hardcoded switches within the source code. There is still an issue about the underlying jpalo library, seems the newer versions don't work properly with the palokettle plugin.
    From our tests we found the jpalo of April works best, the newer ones have HUGE performance issues and sometimes the process starves because the stream is not fast enough pulling in.

    I enclose our latest version of the palokettle plugin and the jpalo we used during our last tests successfully.

    Please download it from:
    mybiq.com/public-files/palo-pe…on-plugin-palokettle/view
    mybiq.com/public-files/jpalo-j…ested-for-kettle-pdi/view

    Thanks to Ben the numeric and string switch works fine - I personally don't see, how people ever could implement and use the plugin beforehand!

    Have fun, Alexander
    Files
  • connection test button works

    ... in the meantime we fixed also this bug and text/strings can be imported/written to palo cubes, like attribute cubes or status etc..

    Pls. download the latest version from mybiq.com/public-files/.

    But there is still an issue with open connections, lazy conversion, the new jpalo library. Fixes will be provided depending on our requirements within projects. Now this published jar file works fine.

    The post was edited 1 time, last by apohl ().

  • Hi Samtek,
    probably you have done sth wrong, it's not really a question of the versions used, beside you must use PDI 3 upwards ...

    But as I pointed out earlier, use the older jpalo API, not the actual one from tensegrity's website. We didn't yet find the bug, which is slowing down the performance on SOME pc's - but not on all ...

    If you download the jpalo and palokettle library and locate them in libext you are ready to go!


    What are your config settings, how is your palo.ini looking like? Do you have a firewall enabled, do you use it on the same pc or remotely?

    cheers alexander
  • HI Alex,
    well I tryed everything what i thought to get it right(different operating systems,different versions of palo and kettel etc) but it didnt work, I used to see the palo tree before, but I dont see it now. And recently while trying to put the jpalo to libext after putting palokettelplugin in libext I get the message that the folder "meta-inf" already exists and if it should be replaced?
    -well my palo configurations are the normal ones i didnt change anything on it because i am letting the server run on a single workstation! and yes I have firewall, but I dont think it is has something to do with the firewall.

    tnks for the help
  • Hi Samtek,

    it was a year ago, when I faced the same problem: no tree-menue in kettle/PDI. And really - I started going mad. I don't work with PDI anymore, but if I remember it correctly, this problem was easy to solve: just create a new project/document and voila ... you will see the tree-menue. It was so simple!
    Hopefully this helps ... just a try!

    cuboo
  • hi samtek,

    i'm working for 5 months with pdi.(and it works really good) :D

    here the version details:

    PDI/KETTLE: version 3.04
    palo plugin: 1.2c
    jpalo api: 2.0

    i tried several other version combinations but this is the only working.

    !! you have to copy (ONLY!!)the 2 jar files into the libext directory of your pentaho extraction folder!! (seems like you copied some folders too)

    greetings

    csae1603
  • hi csae1603 ,
    as you see in the attachments both(the plugin and jpalo) have many folders, and there is no specific jar folder which i could find. but I also think atleast in the case of the palokettelplugin that alle the folders are neccesary because there are many classes in each one, and they wouldnt be there with out use dont you think? and if not in which one is this jar.file? anyways I found on the jpalo website a jpalo.jar file but i still dont get to see the tree*sad* does it have something to do with palo 2.5 maybe? which palo are you using? would it be possible for you to upload yur working pdi here?


    tnks for the help

    samtek
    Images
    • jpalo.jpg

      46.96 kB, 716×416, viewed 2,760 times
    • palokettel.jpg

      45.31 kB, 718×414, viewed 2,715 times

    The post was edited 2 times, last by samtek ().

  • Okay using the palokettel plugin , which alexander posted in this thread and the jpalo on the mybig.com website i get to see the palo tree in the pdi, but i still get the old error messages(attached) when i try to 1) test a database connection to palo demo, 2)configure a palo step(for example palo dimension input) does it have something to do with palo 2.5 which i am running?
    Images
    • fehler.jpg

      129.3 kB, 720×407, viewed 2,658 times
    • fehler2.jpg

      101.54 kB, 650×418, viewed 2,744 times

    The post was edited 1 time, last by samtek ().