hi sentosa,

the kettle plugin works fine with one exception. when i'm importing dimensions in palo with kettle i always get text formatted dimensions, with those i doen't see any data in the cube. (happens always when the fields are defined as string, with integer formatted fields it works properly)

may you can give me an advise how to format or transform the database input files???

greatings
markus

Hi sentosa,

Just like you I'm trying to figure out how to do data import
with
- the Pentaho Data Integration (PDI) - i'm using the new 3.1.0-RC1 link
- with Palo 2.5 (not 2.0!) link
- Palo Java api 2.0 (designed for Palo 2.0) link
- and the latest PaloKettlePlugin 1.0 (designed for Palo 2.0) from Jan 28 2008 link

I do not know which versions you have working, but I'd appreciate some assistance.
my Demo database is up and running
jpalo.jar and PaloKettle.jar are in libext directory of PDI
as the documentation describes.

I'm trying to set up the connection in PDI to the Demo database on localhost as described in the PaloKettePlugin documentation, but get an error message when testing.

(please see images attached)

I have also tried to open the connection from the PaloDimension Output block... but get
an error "Dimension does not exist"
when pressin the "Get Current Levels" button for the block

I'm afraid that this is due to the incompatibility between the PaloKettlePlugin (dated January 2008 and the new 2.5 version of Palo ... Or the new version of Kettle (=PDI)

All Help appreciated - if somebody has figured out how to get Palo 2.5 to work with new 3.0+ version of Kettle.
Does any body know anything about planned future updates to the Kettle plugin???

Thanks
G.

p.s. similar issues have been raised in Pantaho forum
link
The mentioned remedy is to use the OLDER version of Palo Java API 1.5
I don't quite understand why.

The PaloKettlePlugin 1.0 DOES work with
- Pentaho Data Integration (PDI) 3.1.0-RC1
- Palo 2.5
- Palo Java API Build 2.0.0.214

BUT it has the following limitations
- testing the database connection does not work (it always shows an error message), but actual transformations works OK, if the connection is defined correct.
- all elements are hardcoded to be created as strings which seldom are useful, because data in cubes will not be consolidated in dimension hierarchies.

I have made my own version 1.1 of the Plugin which has the following additions:
- option to choose the Default Element Type (Numeric or String) for new elements being loaded into a dimension.
- option to clear (delete) all elements from dimension before loading new elements.
- option to clear (delete) all values from cube before loading new values.

The Test button in the database connection for a Palo database has not been fixed, as I could not figure out how to do this.

It is also not compatible with transformations that contains "Palo Dimension Output" or "Palo Cells Output" steps created with version 1.0 of the plugin.

The attached zip file contains both a ready-to-be-used compiled jar and the source code.

I have also tried to open the connection from the PaloDimension Output block... but get an error "Dimension does not exist" when pressin the "Get Current Levels" button for the block

The Palo Dimension Output step is used for inserting elements into a dimension in your Palo database. I see in your last screenshot that you have selected a Connection "Palo Demo", but you have not selected a dimension in the drop down list just above the "Get Current Levels" button. That is the reason you get the error "Dimension" does not exist".

Kind regards,
Bent

hi bent!

fantastic job!
now pentaho/kettle is the ideal and best ETL-tool for palo at the moment. (for those who want a Graphical User Interface)

great

greatings
markus

Hi all,

I've copied new version (11) you posted here and it seems that it works.
The Sourceforge is not updated to this version or is it the same ?
Many thanks
There is still the connection test error but it's not blocking .
Is that correct ?
The errror:

Errore di connessione al database [classicmodels] : java.lang.InstantiationException: plugin.palo.core.PaloHelper
java.lang.InstantiationException: plugin.palo.core.PaloHelper
at java.lang.Class.newInstance0(Unknown Source)
at java.lang.Class.newInstance(Unknown Source)
at org.pentaho.di.core.database.DatabaseMeta.getDatabaseFactory(DatabaseMeta.java:2495)
at org.pentaho.di.core.database.DatabaseMeta.testConnection(DatabaseMeta.java:2477)
at org.pentaho.di.ui.core.database.dialog.DatabaseDialog.test(DatabaseDialog.java:1919)
at org.pentaho.di.ui.core.database.dialog.DatabaseDialog.test(DatabaseDialog.java:1901)
at org.pentaho.di.ui.core.database.dialog.DatabaseDialog$4.handleEvent(DatabaseDialog.java:293)
at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
at org.pentaho.di.ui.core.database.dialog.DatabaseDialog.open(DatabaseDialog.java:368)
at org.pentaho.di.ui.spoon.delegates.SpoonDBDelegate.newConnection(SpoonDBDelegate.java:505)
at org.pentaho.di.ui.spoon.Spoon.doubleClickedInTree(Spoon.java:2284)
at org.pentaho.di.ui.spoon.Spoon.access$500(Spoon.java:240)
at org.pentaho.di.ui.spoon.Spoon$8.widgetDefaultSelected(Spoon.java:1343)
at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
at org.pentaho.di.ui.spoon.Spoon.readAndDispatch(Spoon.java:873)
at org.pentaho.di.ui.spoon.Spoon.start(Spoon.java:5639)
at org.pentaho.di.ui.spoon.Spoon.run(Spoon.java:5735)
at org.pentaho.di.ui.spoon.Spoon.main(Spoon.java:371)

I've copied new version (11) you posted here and it seems that it works. The Sourceforge is not updated to this version or is it the same ?

I did not make the original version available on sourceforge. I have sent a mail to the people behind the original plugin with my updated version, but I have not received a response yet. So it is still the original version 1.0 that is available on sourceforge.

There is still the connection test error but it's not blocking . Is that correct ?

As I mentioned in my previous post I could not figure out how to fix the Test button, so this does still not work. But transformations will work OK, as long as the connection information are correct.

Hi folks, good news:
We have added some features, like using attribute dimensions and cubes. In addition we have fixed some bugs, like hardcoded switches within the source code. There is still an issue about the underlying jpalo library, seems the newer versions don't work properly with the palokettle plugin.
From our tests we found the jpalo of April works best, the newer ones have HUGE performance issues and sometimes the process starves because the stream is not fast enough pulling in.

I enclose our latest version of the palokettle plugin and the jpalo we used during our last tests successfully.

Please download it from:
mybiq.com/public-files/palo-pe…on-plugin-palokettle/view
mybiq.com/public-files/jpalo-j…ested-for-kettle-pdi/view

Thanks to Ben the numeric and string switch works fine - I personally don't see, how people ever could implement and use the plugin beforehand!

Have fun, Alexander

... in the meantime we fixed also this bug and text/strings can be imported/written to palo cubes, like attribute cubes or status etc..

Pls. download the latest version from mybiq.com/public-files/.

But there is still an issue with open connections, lazy conversion, the new jpalo library. Fixes will be provided depending on our requirements within projects. Now this published jar file works fine.

Now it works really good.
At this point it's strange for me that i had that big problems at the beginning....

Thanks for great help!!!

Stephan

@Schnellerblitz, can yu please tell me which Versions of palo,kettle,jpalo and palokettleplugin you are using, cause I cant somehow get the database connection to start:(

Hi Samtek,
probably you have done sth wrong, it's not really a question of the versions used, beside you must use PDI 3 upwards ...

But as I pointed out earlier, use the older jpalo API, not the actual one from tensegrity's website. We didn't yet find the bug, which is slowing down the performance on SOME pc's - but not on all ...

If you download the jpalo and palokettle library and locate them in libext you are ready to go!


What are your config settings, how is your palo.ini looking like? Do you have a firewall enabled, do you use it on the same pc or remotely?

cheers alexander

HI Alex,
well I tryed everything what i thought to get it right(different operating systems,different versions of palo and kettel etc) but it didnt work, I used to see the palo tree before, but I dont see it now. And recently while trying to put the jpalo to libext after putting palokettelplugin in libext I get the message that the folder "meta-inf" already exists and if it should be replaced?
-well my palo configurations are the normal ones i didnt change anything on it because i am letting the server run on a single workstation! and yes I have firewall, but I dont think it is has something to do with the firewall.

tnks for the help

Hi Samtek,

it was a year ago, when I faced the same problem: no tree-menue in kettle/PDI. And really - I started going mad. I don't work with PDI anymore, but if I remember it correctly, this problem was easy to solve: just create a new project/document and voila ... you will see the tree-menue. It was so simple!
Hopefully this helps ... just a try!

cuboo

hi cuboo,
as you can see in the attached screenshot, i also opened a new transformation, but i dont see the palo tree:( but tnks for the help

hi samtek,

i'm working for 5 months with pdi.(and it works really good)

here the version details:

PDI/KETTLE: version 3.04
palo plugin: 1.2c
jpalo api: 2.0

i tried several other version combinations but this is the only working.

!! you have to copy (ONLY!!)the 2 jar files into the libext directory of your pentaho extraction folder!! (seems like you copied some folders too)

greetings

csae1603

hi csae1603 ,
as you see in the attachments both(the plugin and jpalo) have many folders, and there is no specific jar folder which i could find. but I also think atleast in the case of the palokettelplugin that alle the folders are neccesary because there are many classes in each one, and they wouldnt be there with out use dont you think? and if not in which one is this jar.file? anyways I found on the jpalo website a jpalo.jar file but i still dont get to see the tree*sad* does it have something to do with palo 2.5 maybe? which palo are you using? would it be possible for you to upload yur working pdi here?


tnks for the help

samtek

Okay using the palokettel plugin , which alexander posted in this thread and the jpalo on the mybig.com website i get to see the palo tree in the pdi, but i still get the old error messages(attached) when i try to 1) test a database connection to palo demo, 2)configure a palo step(for example palo dimension input) does it have something to do with palo 2.5 which i am running?